程式扎記: [ 文章收集 ] How To Set Up a Private Docker Registry on Ubuntu 14.04

標籤

2015年9月5日 星期六

[ 文章收集 ] How To Set Up a Private Docker Registry on Ubuntu 14.04

Source From Here
Introduction
Docker is a great tool for deploying your servers. While docker.io lets you upload your Docker creations to their registry for free, anything you upload is also public. This probably isn't what you want for a non-open source-project.

This guide will show you how to set up and secure your own private Docker registry. By the end of this tutorial you will be able to push a custom Docker image to your private registry, and pull the image securely from a different host.

This tutorial doesn't cover containerizing your own application, but only how to create the registry where you can store your deployments. If you want to learn how to get started with Docker itself (as opposed to the registry), you may want to read the tutorial here.

This tutorial has been tested with all servers (one registry and one client) running Ubuntu 14.04, but may work with other Debian-based distros.

Docker Concepts
If you haven't used Docker before then it's worth taking a minute to go through a few of Docker's key concepts. If you're already using Docker and just want to know how to get started running your own registry, then please skip ahead to the next section.

For a refresher on how to use Docker, take a look at the excellent Docker Cheat Sheet here.

Docker at it's core is a way to separate an application and the dependencies needed to run it from the operating system itself. To make this possible Docker uses containers and imagesA Docker image is basically a template for a filesystem. When you run a Docker image with the docker run command, an instance of this filesystem is made live, and runs on your system inside a Docker container. By default this container can't touch the original image itself, or the filesystem of the host where docker is running. It's a self-contained environment.

Whatever changes you make in the container are preserved in that container itself, and don't affect the original image. If you decide you want to keep those changes, then you can "commit" a container to a Docker image (via the docker commit command). This means you can then spawn new containers that start with the contents of your old container, without affecting the original container (or image). If you're familiar with git then the workflow should seem quite similar: you can create new branches (images in Docker parlance) from any container. Running an image is a bit like doing a git checkout.

To continue the analogy, running a private Docker registry is like running a private Git repository for your Docker images.

Step One — Install Prerequisites
You should create a user with sudo access on the registry server (and on the clients when you get that far).

The Docker registry is a Python application, so to get it up and running we need to install the Python development utilities and a few libraries:
$ sudo apt-get update
$ sudo apt-get -y install build-essential python-dev libevent-dev python-pip liblzma-dev
$ sudo apt-get install swig
$ sudo apt-get install libssl-dev

Step Two — Install and Configure Docker Registry
To install the latest stable release of the Docker registry (0.7.3 at the time of writing) we'll use Python's package management utility pip:
$ sudo pip install docker-registry
$ which docker-registry
/usr/local/bin/docker-registry

Docker-registry requires a configuration file.

pipby default installs this config file in a rather obscure location, which can differ depending how your system's Python is installed. So, to find the path, we'll attempt to run the registry and let it complain:
$ sudo /usr/local/bin/docker-registry

Since the config file isn't in the right place yet it will fail to start and spit out an error message that contains a FileNotFoundError that looks like this:
FileNotFoundError: Heads-up! File is missing: /usr/local/lib/python2.7/dist-packages/docker_registry/lib/../../config/config.yml

The registry includes a sample config file called config_sample.yml at the same path, so we can use the path it gave us to locate the sample file. Copy the path from the error message, and use sample config as config.yml portion:
$ ls /usr/local/lib/python2.7/dist-packages/docker_registry/lib/../../config/
boto.cfg config_mirror.yml config_sample.yml
$ cd /usr/local/lib/python2.7/dist-packages/docker_registry/lib/../../config/
$ sudo cp config_sample.yml config.yml
$ docker-registry --help
usage: docker-registry [-h]

run the docker-registry with gunicorn, honoring the following
environment variables:
REGISTRY_HOST: TCP host or ip to bind to; default is 0.0.0.0
REGISTRY_PORT: TCP port to bind to; default is 5000
GUNICORN_WORKERS: number of worker processes gunicorn should start
GUNICORN_GRACEFUL_TIMEOUT: timeout in seconds for graceful worker restart
GUNICORN_SILENT_TIMEOUT: timeout in seconds for restarting silent workers
GUNICORN_USER: unix user to downgrade priviledges to
GUNICORN_GROUP: unix group to downgrade priviledges to
GUNICORN_ACCESS_LOG_FILE: File to log access to
GUNICORN_ERROR_LOG_FILE: File to log errors to
GUNICORN_OPTS: extra options to pass to gunicorn

optional arguments:
-h, --help show this help message and exit

Docker by default saves its data under the /tmp directory, which can lead to unpleasantness since the /tmp folder is cleared on reboot on many flavors of Linux. Let's create a more permanent folder to store our data:
$ sudo mkdir /var/docker-registry

Now we'll edit the config.yml file to update any references to /tmp to /var/docker-registry. First look for a line near the top of the file that starts withsqlalchemy_index_database:


Change it to point to /var/docker-registry like so:
  1. # SQLite search backend  
  2. sqlalchemy_index_database: _env:SQLALCHEMY_INDEX_DATABASE:sqlite:////var/docker-registry/docker-registry.db  
Look a bit further down the file for the local: section, and repeat the process, changing to:
  1. local: &local  
  2.     <<: common="" nbsp="" span="">
  3.     storage: local  
  4.     storage_path: _env:STORAGE_PATH:/var/docker-registry/registry  
The other default values in the sample config are fine, so no need to change anything there. Feel free to look through them. If you want to do something more complex like using external storage for your Docker data, this file is the place to set it up. That's outside the scope of this tutorial though, so you'll have to check the docker-registry documentation if you want to go that route.

Now that the config is in the right place let's try to test the server again:
$ sudo gunicorn --access-logfile - --debug -k gevent -b 0.0.0.0:5000 -w 1 docker_registry.wsgi:application
05/Sep/2015:20:40:14 +0000 WARNING: Cache storage disabled!
05/Sep/2015:20:40:14 +0000 WARNING: LRU cache disabled!
05/Sep/2015:20:40:14 +0000 DEBUG: Will return docker-registry.drivers.file.Storage

Great! Now we have a Docker registry running. Go ahead and kill it with Ctrl+C.

At this point the registry isn't that useful yet — it won't start unless you type in the above gunicorn command. Also, Docker registry doesn't come with any built-in authentication mechanism, so it's insecure and completely open to the public right now.

Step Three - Start Docker Registry as a Service
Let's set the registry to start on system startup by creating an Upstart script.

First let's create a directory for the log files to live in:
$ sudo mkdir -p /var/log/docker-registry

Then use your favorite text editor to create an Upstart script:
$ sudo vi /etc/init/docker-registry.conf
  1. description "Docker Registry"  
  2.   
  3. start on runlevel [2345]  
  4. stop on runlevel [016]  
  5.   
  6. respawn  
  7. respawn limit 10 5  
  8.   
  9. script  
  10. exec gunicorn --access-logfile /var/log/docker-registry/access.log --error-logfile /var/log/docker-registry/server.log -k gevent --max-requests 100 --graceful-timeout 3600 -t 3600 -b localhost:5000 -w 8 docker_registry.wsgi:application  
  11. end script  

For more about Upstart scripts, please read this tutorial.

Now it is time to start docker registry service:
$ service docker-registry status
docker-registry stop/waiting
$ sudo service docker-registry start
docker-registry start/running, process 32735
$ tail /var/log/docker-registry/server.log
[2015-09-05 21:41:45 +0000] [32735] [INFO] Listening at: http://127.0.0.1:5000 (32735)
[2015-09-05 21:41:45 +0000] [32735] [INFO] Using worker: gevent
[2015-09-05 21:41:45 +0000] [32741] [INFO] Booting worker with pid: 32741
[2015-09-05 21:41:45 +0000] [32744] [INFO] Booting worker with pid: 32744
[2015-09-05 21:41:45 +0000] [32745] [INFO] Booting worker with pid: 32745
...

Now that the server's running in the background, let's move on to configuring Nginx so the registry is secure.

Step Four — Secure Your Docker Registry with Nginx
The first step is to set up authentication so that not just anybody can log into our server. Let's install Nginx and the apache2-utils package (which allows us to easily create authentication files that Nginx can read).
$ sudo apt-get -y install nginx apache2-utils

Now it's time to create our Docker users. Create the first user as follows:
$ sudo htpasswd -c /etc/nginx/docker-registry.htpasswd duser
New password: 
Re-type new password: 
Adding password for user duser

Create a new password for this user when prompted. If you want to add more users in the future, just re-run the above command without the c option:
$ sudo htpasswd /etc/nginx/docker-registry.htpasswd ANOTHER_USER

At this point we have a docker-registry.htpasswd file with our users set up, and a Docker registry available. You can take a peek at the file at any point if you want to view your users (and remove users if you want to revoke access).

Next we need to tell Nginx to use that authentication file, and to forward requests to our Docker registry. Let's create an Nginx configuration file. Create a newdocker-registry file, entering your sudo password if needed:
# sudo vi /etc/nginx/sites-available/docker-registry
  1. # For versions of Nginx > 1.3.9 that include chunked transfer encoding support  
  2. # Replace with appropriate values where necessary  
  3.   
  4. upstream docker-registry {  
  5. server localhost:5000;  
  6. }  
  7.   
  8. server {  
  9. listen 8080;  
  10. server_name my.docker.registry.com;  
  11.   
  12. # ssl on;  
  13. # ssl_certificate /etc/ssl/certs/docker-registry;  
  14. # ssl_certificate_key /etc/ssl/private/docker-registry;  
  15.   
  16. proxy_set_header Host       $http_host;   # required for Docker client sake  
  17. proxy_set_header X-Real-IP  $remote_addr; # pass on real client IP  
  18.   
  19. client_max_body_size 0; # disable any limits to avoid HTTP 413 for large image uploads  
  20.   
  21. # required to avoid HTTP 411: see Issue #1486 (https://github.com/dotcloud/docker/issues/1486)  
  22. chunked_transfer_encoding on;  
  23.   
  24. location / {  
  25.      # let Nginx know about our auth file  
  26.      auth_basic              "Restricted";  
  27.      auth_basic_user_file    docker-registry.htpasswd;  
  28.   
  29.      proxy_pass http://docker-registry;  
  30. }  
  31. location /_ping {  
  32.      auth_basic off;  
  33.      proxy_pass http://docker-registry;  
  34. }    
  35. location /v1/_ping {  
  36.      auth_basic off;  
  37.      proxy_pass http://docker-registry;  
  38. }  
  39.   
  40. }  

And link it up so that Nginx can use it:
$ sudo ln -s /etc/nginx/sites-available/docker-registry /etc/nginx/sites-enabled/docker-registry

Then restart Nginx to activate the virtual host configuration:
$ sudo service nginx restart
* Restarting nginx nginx [ok]

Let's make sure everything worked. Our Nginx server is listening on port 8080, while our original docker-registry server is listening on localhost port 5000. We can use curl to see if everything is working:
$ curl localhost:5000
"\"docker-registry server\""
$ curl localhost:8080

401 Authorization Required

401 Authorization Required



nginx/1.4.6 (Ubuntu)
  

It's worthwhile to run these two test commands from a remote machine as well, using the server's IP address instead of localhost, to verify that your ports are set up correctly. In the Upstart config file we told docker-registry to listen only on localhost, which means it shouldn't be accessible from the outside on port 5000. Nginx, on the other hand, is listening on port 8080 on all interfaces, and should be accessible from the outside. If it isn't then you may need to adjust your firewall permissions.

Good, so authentication is up. Let's try to log in now with one of the usernames you created earlier:
$ curl duser:yourpassword@localhost:8080 // Change 'yourpassword' to real password set before!
"\"docker-registry server\""

Step Five — Set Up SSL
At this point we have the registry up and running behind Nginx with HTTP basic authentication working. However, the setup is still not very secure since the connections are unencrypted. You might have noticed the commented-out SSL lines in the Nginx config file we made earlier.

Let's enable them. First, open the Nginx configuration file for editing:
$ sudo vi /etc/nginx/sites-available/docker-registry
  1. ...  
  2. server {  
  3. listen 8080;  
  4. server_name yourdomain.com;  
  5.   
  6. ssl on;  
  7. ssl_certificate /etc/ssl/certs/docker-registry;  
  8. ssl_certificate_key /etc/ssl/private/docker-registry;  
  9. ...  

Save the file. Nginx is now configured to use SSL and will look for the SSL certificate and key files at /etc/ssl/certs/docker-registry and /etc/ssl/private/docker-registry respectively.

If you already have an SSL certificate set up or are planning to buy one, then you can just copy the certificate and key files to the paths listed above (ssl_certificate and ssl_certificate_key); or you could also get a free signed SSL certificate.

Or, use a self-signed SSL certificate. Since Docker currently doesn't allow you to use self-signed SSL certificates this is a bit more complicated than usual, since we'll also have to set up our system to act as our own certificate signing authority.

Signing Your Own Certificate
First let's make a directory to store the new certificates and go there:
$ mkdir ~/certs
$ cd ~/certs/
$ openssl genrsa -out devdockerCA.key 2048 // Generate a new root key
Generating RSA private key, 2048 bit long modulus
..+++
.......................................+++
e is 65537 (0x10001)

$ openssl req -x509 -new -nodes -key devdockerCA.key -days 10000 -out devdockerCA.crt // Generate a root certificate
// Then generate a key for your server (this is the file we'll later copy to /etc/ssl/private/docker-registry for Nginx to use):
$ openssl genrsa -out dev-docker-registry.com.key 2048

Now we have to make a certificate signing request.

After you type this command OpenSSL will prompt you to answer a few questions. Write whatever you'd like for the first few, but when OpenSSL prompts you to enter the "Common Name" make sure to type in the domain of your server.
$ openssl req -new -key dev-docker-registry.com.key -out dev-docker-registry.com.csr

Do not enter a challenge password. Then we need to sign the certificate request:
$ openssl x509 -req -in dev-docker-registry.com.csr -CA devdockerCA.crt -CAkey devdockerCA.key -CAcreateserial -out dev-docker-registry.com.crt -days 10000

Now that we've generated all the files we need for our certificate to work, we need to copy them to the correct places.

First copy the certificate and key to the paths where Nginx is expecting them to be:
$ sudo cp dev-docker-registry.com.crt /etc/ssl/certs/docker-registry
$ sudo cp dev-docker-registry.com.key /etc/ssl/private/docker-registry

Since the certificates we just generated aren't verified by any known certificate authority (e.g., VeriSign), we need to tell any clients that are going to be using this Docker registry that this is a legitimate certificate. Let's do this locally so that we can use Docker from the Docker registry server itself:
$ sudo mkdir /usr/local/share/ca-certificates/docker-dev-cert
$ sudo cp devdockerCA.crt /usr/local/share/ca-certificates/docker-dev-cert/
$ sudo update-ca-certificates

SSL Test
Let's restart Nginx to reload the configuration and SSL keys:
$ sudo service nginx restart

Do another curl test (only this time using https) to verify that our SSL setup is working properly. Keep in mind that for SSL to work correctly you will have to use the same domain name you typed into the Common Name field earlier while you were creating your SSL certificate.
$ curl https://duser:yourpassword@docker:8080
"\"docker-registry server\""

Now we have a Docker registry running behind an Nginx server which is providing authentication and encryption via SSL.

Step Six — Access Your Docker Registry from Another Machine
To access your Docker registry, first add the SSL certificate you created earlier to the new client machine. The file you want is located at ~/certs/devdockerCA.crt. You can copy it to the new machine directly or use the below instructions to copy and paste it. On the registry server, view the certificate:
$ cat ~/certs/devdockerCA.crt
  1. -----BEGIN CERTIFICATE-----  
  2. MIIDXTCCAkWgAwIBAgIJANiXy7fHSPrmMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV  
  3. BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX  
  4. aWRnaXRzIFB0eSBMdGQwHhcNMTQwOTIxMDYwODE2WhcNNDIwMjA2MDYwODE2WjBF  
  5. MQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50  
  6. ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB  
  7. CgKCAQEAuK4kNFaY3k/0RdKRK1XLj9+IrpR7WW5lrNaFB0OIiItHV9FjyuSWK2mj  
  8. ObR1IWJNrVSqWvfZ/CLGay6Lp9DJvBbpT68dhuS5xbVw3bs3ghB24TntDYhHMAc8  
  9. GWor/ZQTzjccHUd1SJxt5mGXalNHUharkLd8mv4fAb7Mh/7AFP32W4X+scPE2bVH  
  10. OJ1qH8ACo7pSVl1Ohcri6sMp01GoELyykpXu5azhuCnfXLRyuOvQb7llV5WyKhq+  
  11. SjcE3c2C+hCCC5g6IzRcMEg336Ktn5su+kK6c0hoD0PR/W0PtwgH4XlNdpVFqMST  
  12. vthEG+Hv6xVGGH+nTszN7F9ugVMxewIDAQABo1AwTjAdBgNVHQ4EFgQULek+WVyK  
  13. dJk3JIHoI4iVi0FPtdwwHwYDVR0jBBgwFoAULek+WVyKdJk3JIHoI4iVi0FPtdww  
  14. DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAkignESZcgr4dBmVZqDwh  
  15. YsrKeWSkj+5p9eW5hCHJ5Eg2X8oGTgItuLaLfyFWPS3MYWWMzggxgKMOQM+9o3+k  
  16. oH5sUmraNzI3TmAtkqd/8isXzBUV661BbSV0obAgF/ul5v3Tl5uBbCXObC+NUikM  
  17. O0C3fDmmeK799AM/hP5CTDehNaFXABGoVRMSlGYe8hZqap/Jm6AaKThV4g6n4F7M  
  18. u5wYtI9YDMsxeVW6OP9ZfvpGZW/n/88MSFjMlBjFfFsorfRd6P5WADhdfA6CBECG  
  19. LP83r7/MhqO06EOpsv4n2CJ3yoyqIr1L1+6C7Erl2em/jfOb/24y63dj/ATytt2H  
  20. 6g==  
  21. -----END CERTIFICATE-----  

Copy that output to your clipboard and connect to your client machine. On the client server, create the certificate directory:
$ sudo mkdir /usr/local/share/ca-certificates/docker-dev-cert
// Open the certificate file for editing and paste the certificate contents.
$ vi /usr/local/share/ca-certificates/docker-dev-cert/devdockerCA.crt
$ sudo update-ca-certificates // Now update the certificates

If you don't have Docker installed on the client yet, do so now.

On most versions of Ubuntu you can quickly install a recent version of Docker by following the next few commands. If your client is on a different distro or you have issues then see Docker's installation documentation for other ways to install Docker.

Add the repository key:
$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9;

Create a file to list the Docker repository:
$ sudo vi /etc/apt/sources.list.d/docker.list
  1. deb https://get.docker.io/ubuntu docker main  

Update your package lists:
$ sudo apt-get update
$ sudo apt-get install linux-image-generic-lts-trusty
$ sudo apt-get install curl
$ sudo reboot

Install Docker:
$ curl -sSL https://get.docker.com/ | sh

To make working with Docker a little easier, let's add our current user to the Docker group and re-open a new shell:
$ sudo gpasswd -a ${USER} docker
$ sudo su -l $USER #(enter your password at the prompt if needed)

Restart Docker to make sure it reloads the system's CA certificates.
$ sudo service docker restart

You should now be able to log in to your Docker registry from the client machine:
$ docker login https://YOUR-HOSTNAME:8080

Note that you're using https:// and port 8080 here. Enter the username and password you set up earlier (enter whatever you'd like for email if prompted). You should see a Login Succeeded message. At this point your Docker registry is up and running! Let's make a test image to push to the registry.

Step Seven — Publish to Your Docker Registry
On the client server, create a small empty image to push to our new registry.
$ docker run -t -i ubuntu /bin/bash

After it finishes downloading you'll be inside a Docker prompt. Let's make a quick change to the filesystem:
$ touch /SUCCESS
$ exit // Exit out of the Docker container

Commit the change:
$ docker commit $(docker ps -lq) test-image

If you run docker images now, you'll see that you have a new test-image in the image list:
$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
test-image latest 1f3ce8008165 9 seconds ago 192.7 MB
...

This image only exists locally right now, so let's push it to the new registry we've created. First, log in to the registry with Docker. Note that you want to usehttps:// and port 8080:
$ docker login https://<YOUR-DOMAIN>:8080 // Enter the username and password you set up earlier

Docker has an unusual mechanism for specifying which registry to push to. You have to tag an image with the private registry's location in order to push to it. Let's tag our image to our private registry:
$ docker tag test-image YOUR-DOMAIN:8080/test-image

Note that you are using the local name of the image first, then the tag you want to add to it. The tag is not using https://, just the domain, port, and image name. Now we can push that image to our registry. This time we're using the tag name only:
$ docker push :8080/test-image

This will take a moment to upload to the registry server. You should see output that includes Image successfully pushed.

Step Eight - Pull from Your Docker Registry
To make sure everything worked let's go back to our original server (where you installed the Docker registry) and pull the image we just pushed from the client. You could also test this from a third server.

If Docker is not installed on your test pull server, go back and follow the installation instructions (and if it's a third server, the SSL instructions) from Step Six. Log in with the username and password you set up previously.
$ docker login https://<YOUR-DOMAIN>:8080

And now pull the image. You want just the "tag" image name, which includes the domain name, port, and image name (but not https://):
$ docker pull <YOUR-DOMAIN>:8080/test-image

Docker will do some downloading and return you to the prompt. If you run the image on the new machine you'll see that the SUCCESS file we created earlier is there:
$ docker run -t -i <YOUR-DOMAIN>:8080/test-image /bin/bash
$ ls // You should see the SUCCESS file we created earlier

Congratulations! You've just used your own private Docker registry to push and pull your first Docker container! Happy Docker-ing!

Supplement
Docker Practice - Repository
How do I set my DNS on Ubuntu 14.04?
access private registry: x509: certificate signed by unknown authority
Using option --insecure-registry while starting docker daemon. For example:
# docker daemon --insecure-registry dev.registry.com:5000


沒有留言:

張貼留言

網誌存檔

關於我自己

我的相片
Where there is a will, there is a way!