Hi! Don’t know if you ever figured out a viable solution to your migration. But I can share the following. Directly connecting the the DB container in EE4 would require some additional manual configuration steps and is NOT recommended. Instead consider one of the three following methods:
The Wordpress Way - Use one of the many plugins. I have always preferred WP Migrate Pro by Deliciousbrains, but there are many good plugins that will connect, copy, sync or move your site and its data. If you are looking for a free plugin, check out All in one migrator or duplicator plugins.
The phpMyAdmin way - In EE4 you can install admin tools by issuing the following command
ee admin-tools enable example.com
Once installed you will be able to access phpMyAdmin and using it to import the database you exported from your current live setup. See all the necessary details at https://easyengine.io/handbook/admin-tools/
The Wordpress command line way - Use wp-cli to export and import the database from your current to your new sites. This can be the most confusing method for those unfamiliar with wp-cli and using EE4, especially in the Docker environment. Typically what I do is rsync the exported database from the current site into my users home directory in the new site. You then need to move the file into the Docker container’s volume where your docroot (htdocs folder) is stored. You can usually find this by typing sudo ee site info sitename.com and finding the site root for your site. The app folder that is inside that directory is your docroot and where you need to move the db file. Then you need to go to the shell for that site (sudo ee shell sitename.com) and execute the wp import filename.sql to import your database file.
**NOTE only use the wp-cli tools if you are comfortable.