We already saw how to remove the .html extension from Magento.
When you install Magento probably your URLs would be like:
There is an option to remove that index.php from the URL.
Go to your admin panel -> System -> General -> Web -> Search Engines Optimization.
Change Use Web Server Rewrites to Yes.
Now your URL will be cool enough 😉