I was looking into it and tried to copy paste the below lines in config.inc.php in the phpmyadmin root folder again and again till the number of hosts i wanted to access through PHPMyAdmin.
I Just changed the value of $cfg['Servers'][$i]['host'] to the different host-names in each copy (you can also change the comments for your reference).
/*
* First server
*/
$i++;
/* Authentication type */
$cfg['Servers'][$i]['auth_type'] = 'cookie';
/* Server parameters */
$cfg['Servers'][$i]['host'] = 'localhost'; // change this to your host name of DB
$cfg['Servers'][$i]['connect_type'] = 'tcp';
$cfg['Servers'][$i]['compress'] = false;
/* Select mysql if your server does not have mysqli */
$cfg['Servers'][$i]['extension'] = 'mysqli';
$cfg['Servers'][$i]['AllowNoPassword'] = false;
