Hi,
you can increase performance for example implement caching mechanism into your setup.
For example apc to cache php scripts (the scripts are only compiled once and then executed from the cache). Depending on what php application you are using you might further optimize performance with memcache or other object storage for php sessions.
You can use other caching mechanisms for example Varnish in front of your webserver (which i guess is either Apache or Nginx). This would cache website content to be cached in memory so no php processing is needed until the cache is reset.
You may also put a SSD for database files into you server to further increase read speed which ultimately leads to a faster loading of the website.
Another option is to put static assets, eg images, "into the cloud" such as amazon S3, cloudfront or any other CDN, to off-load traffic from your server.
The next step would be to setup a loadbalancer and add another server with the same configuration. You then need a way to share local files. that could be done via NFS from server A to B, if you only want to use 2 or at maximum 3 servers.
If you want a high scalable (1 + n numbers of servers) you'd need a load balancer, dedicated shared, high available storage (eg. NFS) and database (mysql, memcache, mongo, ...) servers.
2 dedicated hardware servers with virtualized servers performing those tasks would be sufficient for ten thousands of concurrent users
I'd be happy to help you on a long term basis