{"id":129,"date":"2020-03-28T20:50:17","date_gmt":"2020-03-28T17:50:17","guid":{"rendered":"http:\/\/mbrdancer.ru\/?p=129"},"modified":"2020-09-26T23:54:57","modified_gmt":"2020-09-26T20:54:57","slug":"customizing-nvidia-tensorflow-docker-image","status":"publish","type":"post","link":"http:\/\/mbrdancer.ru\/?p=129","title":{"rendered":"customizing NVIDIA tensorflow docker image"},"content":{"rendered":"\n<p>(0) configuring users and environment<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># launch the container from the original NVIDIA image\n\n# in the container launched:\n$ apt-get update\n$ apt-get install -y sudo openssh-server vim screen mc<\/code><\/pre>\n\n\n\n<p>(0.1) Configure sshd using <code>\/etc\/ssh\/sshd_config<\/code> file. e.g., one may use the following list of parameters:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Port 2224\nAddressFamily any\nListenAddress 0.0.0.0\n\nPermitRootLogin no\nPubkeyAuthentication yes\nHostbasedAuthentication no\nIgnoreRhosts yes\nPasswordAuthentication no\nPermitEmptyPasswords no\nChallengeResponseAuthentication no\nUsePAM yes\nAllowTcpForwarding yes\nX11Forwarding yes\nX11UseLocalhost yes\nPermitTTY yes\nPrintMotd no\nTCPKeepAlive yes\nPermitTunnel no\nAcceptEnv LANG LC_*\nSubsystem       sftp    \/usr\/lib\/openssh\/sftp-server<\/code><\/pre>\n\n\n\n<p>(0.2) now configure the user:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>$ groupadd -g 1001 &lt;groupname>\n# 1001 is a group ID\n$ useradd -c 'user description' -u 1001 -g &lt;groupname> -m -d \/home\/&lt;uname> -s \/bin\/bash &lt;uname>\n\n# setting the new user password\n$ passwd &lt;uname>\n# type a password twice. actually, with the preferences mentioned above, the user will not be asked for a password\n\n$ cp \/root\/.bashrc \/home\/&lt;uname>\/\n$ chown &lt;uname>:&lt;gname> \/home\/&lt;uname>\/.bashrc\n$ cp \/root\/.profile \/home\/&lt;uname>\/\n$ chown &lt;uname>:&lt;gname> \/home\/&lt;uname>\/.profile<\/code><\/pre>\n\n\n\n<p>make the new user a sudoer:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>$ visudo\n# add a line like this:\n&lt;uname> ALL=(ALL) NOPASSWD:ALL<\/code><\/pre>\n\n\n\n<p>(0.3) now add environment variables to the <code>.bashrc<\/code>:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># at the host\n$ docker inspect &lt;original_container_name>:&lt;original_container_tag>\n# find the \"Env\" section and copy all the vars<\/code><\/pre>\n\n\n\n<p>copy the contents of the <code>Env<\/code> section and paste it to the file <code>\/home\/&lt;uname&gt;\/.bashrc<\/code> preceding every line with the <code>export <\/code> token<\/p>\n\n\n\n<p>Commit changes to the container and relaunch it from behalf of a new user:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># at the host\n# list all the containers launched\n$ docker ps -a\n\n# list all the images at the host:\n$ docker images\n\n# commit changes:\n$ docker commit &lt;container_name_or_id> &lt;new_image_name>:&lt;new_image_tag>\n\n# in the container:\n$ exit\n\n# at the host:\n$ docker run --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -it --rm -v \/src\/volume\/mountpoint:\/container\/volume\/mountpoint -u &lt;uid>:&lt;gid> -p 2223:2223 &lt;new_image_name>:&lt;new_image_tag> \/bin\/bash<\/code><\/pre>\n\n\n\n<p><\/p>\n\n\n\n<p>find the public key of the new user and import its contents to the <code>authorized_keys<\/code> file:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># in the container launched\n$ cd\n$ mkdir .ssh\n$ chmod 700 .ssh\n$ cd .ssh\n$ cat \/path\/to\/the\/public\/key\/file\/user_docker_TESLA.pub >> authorized_keys\n$ chmod 600 authorized_keys<\/code><\/pre>\n\n\n\n<p>renew host keys:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># in the container\n$ sudo rm -rvf \/etc\/ssh\/ssh_host_*\n$ sudo dpkg-reconfigure openssh-server\n\n# a fix in Ubuntu 16.04\n$ sudo mkdir -p -m 0755 \/var\/run\/sshd<\/code><\/pre>\n\n\n\n<p>commit changes, relaunch the container <\/p>\n\n\n\n<p>(1) installing basemap.<\/p>\n\n\n\n<p>(1.1) installing <a href=\"https:\/\/trac.osgeo.org\/geos\/wiki\/BuildingOnUnixWithCMake\">GEOS<\/a><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># ensure cmake is installed\n$ sudo apt-get update\n$ sudo apt-get install -y cmake\n\n$ git clone https:\/\/git.osgeo.org\/gitea\/geos\/geos.git\n$ cd geos\n$ mkdir build\n$ cd build\n$ cmake ..\/\n$ make\n$ sudo make install<\/code><\/pre>\n\n\n\n<p>(1.2) installing PROJ<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\n$ sudo -E apt-get install proj-bin\n<\/pre><\/div>\n\n\n<p>ensure environment variables point to the right places (refer to the <a href=\"https:\/\/matplotlib.org\/basemap\/users\/installing.html\">basemap docs<\/a>)<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\nexport PROJ_LIB=\/usr\/share\/proj\nexport GEOS_DIR=\/usr\/local\n<\/pre><\/div>\n\n\n<p>(1.3) installing basemap itself<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\n$ git clone https:\/\/github.com\/matplotlib\/basemap.git\n$ cd basemap\n$ sudo -E python setup.py install\n<\/pre><\/div>\n\n\n<p style=\"color:#a4a4a4\" class=\"has-text-color\">[DEPRECATED]<br>(1.3.a) according to <a href=\"https:\/\/github.com\/matplotlib\/basemap\/issues\/456\">this issue<\/a>, we now have to remove the file <code>__init__.py<\/code> from the directory <code>\/usr\/local\/lib\/python3.6\/dist-packages\/basemap-1.2.1-py3.6-linux-x86_64.egg\/mpl_toolkits\/<\/code><\/p>\n\n\n\n<p>(1.3.b) according to this issue, we now should ensure the <a href=\"https:\/\/github.com\/matplotlib\/basemap\/pull\/466\/commits\/ac2957d4ffb78307ecef8a7e6c1bab7f9fea0ea7\">workaround<\/a> is applied in both the files:<br><code>\/usr\/local\/lib\/python3.6\/dist-packages\/basemap-1.2.1-py3.6-linux-x86_64.egg\/mpl_toolkits\/basemap\/proj.py<\/code><br>and<br><code>\/usr\/local\/lib\/python3.6\/dist-packages\/basemap-1.2.1-py3.6-linux-x86_64.egg\/mpl_toolkits\/basemap\/__init__.py<\/code><\/p>\n\n\n\n<p>In particular, the line<br><code>from matplotlib.cbook import dedent<\/code><br>is replaced with the block:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: python; title: ; notranslate\" title=\"\">\ntry:\n    from inspect import cleandoc as dedent\nexcept ImportError:\n    # Deprecated as of version 3.1. Not quite the same\n    # as textwrap.dedent.\n    from matplotlib.cbook import dedent\n<\/pre><\/div>\n\n\n<pre class=\"wp-block-code\"><code>$ sudo mv \/usr\/local\/lib\/python3.6\/dist-packages\/basemap-1.2.1-py3.6-linux-x86_64.egg\/mpl_toolkits\/__init__.py \/usr\/local\/lib\/python3.6\/dist-packages\/basemap-1.2.1-py3.6-linux-x86_64.egg\/mpl_toolkits\/__init__.py.bak<\/code><\/pre>\n\n\n\n<p>(2) install other packages<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>$ sudo -E pip install opencv-python imgaug scikit-image imageio pandas scikit-learn netcdf4\n$ sudo -E apt-get install -y libsm6 libxext6 libxrender-dev libgl1-mesa-glx<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>(0) configuring users and environment (0.1) Configure sshd using \/etc\/ssh\/sshd_config file. e.g., one may use the following list of parameters: (0.2) now configure the user: make the new user a sudoer: (0.3) now add environment variables to the .bashrc: copy the contents of the Env section and paste it to the file \/home\/&lt;uname&gt;\/.bashrc preceding every&hellip; <a class=\"more-link\" href=\"http:\/\/mbrdancer.ru\/?p=129\">\u0427\u0438\u0442\u0430\u0442\u044c \u0434\u0430\u043b\u0435\u0435 <span class=\"screen-reader-text\">customizing NVIDIA tensorflow docker image<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"_links":{"self":[{"href":"http:\/\/mbrdancer.ru\/index.php?rest_route=\/wp\/v2\/posts\/129"}],"collection":[{"href":"http:\/\/mbrdancer.ru\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/mbrdancer.ru\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/mbrdancer.ru\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/mbrdancer.ru\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=129"}],"version-history":[{"count":24,"href":"http:\/\/mbrdancer.ru\/index.php?rest_route=\/wp\/v2\/posts\/129\/revisions"}],"predecessor-version":[{"id":155,"href":"http:\/\/mbrdancer.ru\/index.php?rest_route=\/wp\/v2\/posts\/129\/revisions\/155"}],"wp:attachment":[{"href":"http:\/\/mbrdancer.ru\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=129"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/mbrdancer.ru\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=129"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/mbrdancer.ru\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=129"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}