Why mirroring Magento Marketplace?

When you are installing Magento 2 via composer, you will be downloading packages from Marketplace: This can be slow at times. Setting up a local server with all Magento packages will make deployment of new development sites a lot faster: Packages are only synced once with Marketplace and then bandwidth is reduced to local traffic between you and your mirror.

Besides speed, theoretically, the Marketplace could be down, theoretically. Depending on the Marketplace servers will also bring up the chance that you going live with certain changes is impossible due to composer dependencies, if these servers are down. Having your own server might reduce this risk, though it obviously introduces another risk: That your own server is down, theoretically.

Quick walk-through

So let's go through the steps I took to get going with my own mirror. Again, all kudos go to Alan Storm. Most importantly, you will need Satis, a package that allows you to setup your own composer repository (your own Packagist), which comes in handy if you want to serve your own private packages. Satis ships with its own web-interface plus a command-line tool bin/satis to manage things. Here it will be used to mirror Marketplace.

Installing Satis

First of all, I created a new site folder (the project root) with a public folder that will host the Satis webpage. Satis itself is installed in the project root (or actually a subfolder satis) and specifically not in the world-accessible public folder for security reasons. Checking out the Satis sources through git looks like this (I usually prefer to check out sources via Git / SSH because of version management, instead of using the plain HTTP link):

git clone git@github.com:composer/satis.git

Next, Satis still requires its own dependencies which are installed via composer:

cd satis/
composer install

Configuring Satis

Let's get down to the actual configuration: Satis reads its configuration from a file satis.json which I stored in my site-folder (so not the satis/ folder, not the public/ folder, but the root of my project). The file looks as follows:

{
    "name": "My mirror of repo.magento.com",
    "homepage": "https://my-mirror.yireo.com",
    "repositories": [
        {
            "type": "composer",
            "url": "https://repo.magento.com"
        }
    ],
    "require-dependencies": true,
    "require-dev-dependencies": true,
    "require-all": true,
    "archive": {
        "directory": "dist",
        "format": "zip",
        "prefix-url": "https://my-mirror.yireo.com",
        "skip-dev": false
    }
}

Obviously, replace the URL with your own. Now we can call upon Satis to build all Satis files using this configuration. Because there's quite a bunch of Magento packages out there, and because I'm not willing to simply increase the PHP memory_limit globally, I add a flag memory_limit=1G to the command-line:

php -d memory_limit=1G satis/bin/satis build satis.json public

The command will take some time to complete. Once finished, you can finalize your webserver configuration, create a new Virtual Host that has its root pointing to public. Accessing the URL should show you a listing of Satis-mirrored Magento packages.

Small note: Use SSL

In the example above I've used a custom URL with SSL enabled. Make sure to use SSL. Composer will start complaining if there is no SSL certificate (or a wrong SSL certificate) for your mirror. You can disable the check (secure-http set to false) but I strongly recommend you to use LetsEncrypt to get a free certificate. This gives its own set of issues when the server is not web-accessible (though using LetsEncrypt is still an option here), but you can expect non-SSL to become obsolete in the upcoming years.

Additionally, use HTTP/2 on your webserver. Composer clients support it. And it is fast. You should run HTTP/2 everywhere. Period.

Usage for installing Magento 2

To install Magento 2, the following composer command can now be used:

composer create-project --prefer-dist --no-progress --repository-url=https://my-mirror.yireo.com magento/project-community-edition .

My own findings are that a Magento 2 install goes a lot quicker through your own Satis mirror, than it would by using the Magento servers.

Marketplace authentication

When you have never used composer to fetch packages from the Magento Marketplace, composer will ask you for your Marketplace credentials. You could enter these manually with Satis as well. However, I recommend you to setup a .composer/auth.json file within the homefolder of the user that runs the Satis commands, so that credentials are found automatically:

{
    "github-oauth": {
        "github.com": "MY_GITHUB_KEY"
    },
    "http-basic": {
        "repo.magento.com": {
            "username": "MY_MARKETPLACE_USERNAME",
            "password": "MY_MARKETPLACE_PASSWORD"
        }
    }
}

I add in a GitHub API key as well, just to make sure that GitHub rate limiting doesn't bite me.

If you want to place your authentication file elsewhere, you can change the COMPOSER_HOME variable specifically for the satis command:

COMPOSER_HOME=/abc php -d memory_limit=1G satis/bin/satis build satis.json public 

Downsides of mirroring Marketplace

The main downside of mirroring Marketplace yourself is that you have to keep it in sync. This could be fixed by running a script through cron, but still, it is a burden. To update things, simply repeat the build command as mentioned above. Other than that, I hoped you found this guide easy to follow. Composer downloads can be as fast as you make them.