Skip to content

Commit 5e38101

Browse files
mathbunnyrumathbunnyru
authored
Move recipe Dockerfiles to separate files (#1953)
1 parent bc938c5 commit 5e38101

13 files changed

+128
-100
lines changed

‎.hadolint.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,4 @@
22
ignored:
33
- DL3006
44
- DL3008
5+
- DL3013

‎.pre-commit-config.yaml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,16 @@ repos:
7272
- id: hadolint-docker
7373
entry: hadolint/hadolint:v2.12.1-beta hadolint
7474

75+
# Lint: Dockerfile
76+
# We're linting .dockerfile files as well
77+
- repo: https://github.com/hadolint/hadolint.git
78+
rev: v2.12.1-beta
79+
hooks:
80+
- id: hadolint-docker
81+
entry: hadolint/hadolint:v2.12.1-beta hadolint
82+
types: [file]
83+
files: \.dockerfile$
84+
7585
# Lint: YAML
7686
- repo: https://github.com/adrienverge/yamllint.git
7787
rev: v1.32.0

‎docs/contributing/lint.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@ The following rules are ignored by default for all images in the `.hadolint.yaml
5050
- `base-notebook` `FROM` clause is fixed but based on an argument (`ARG`).
5151
- Building downstream images from (`FROM`) the latest is done on purpose.
5252
- [`DL3008`][dl3008]: System packages are always updated (`apt-get`) to the latest version.
53+
- [`DL3013`][dl3013]: We always install latest packages using `pip`
5354

5455
The preferred way to do it for other rules is to flag ignored ones in the `Dockerfile`.
5556

@@ -69,4 +70,5 @@ RUN cd /tmp && echo "hello!"
6970
[rules]: https://github.com/hadolint/hadolint#rules
7071
[dl3006]: https://github.com/hadolint/hadolint/wiki/DL3006
7172
[dl3008]: https://github.com/hadolint/hadolint/wiki/DL3008
73+
[dl3013]: https://github.com/hadolint/hadolint/wiki/DL3013
7274
[pre-commit]: https://pre-commit.com/

‎docs/using/recipe_code/dask_jupyterlab.dockerfile

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
FROM jupyter/base-notebook
2+
3+
# Install the Dask dashboard
4+
RUN mamba install --yes 'dask-labextension' && \
5+
mamba clean --all -f -y && \
6+
fix-permissions "${CONDA_DIR}" && \
7+
fix-permissions "/home/${NB_USER}"
8+
9+
# Dask Scheduler port
10+
EXPOSE 8787

‎docs/using/recipe_code/jupyterhub_version.dockerfile

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
FROM jupyter/base-notebook
2+
3+
RUN mamba install --yes 'jupyterhub==4.0.1' && \
4+
mamba clean --all -f -y && \
5+
fix-permissions "${CONDA_DIR}" && \
6+
fix-permissions "/home/${NB_USER}"

‎docs/using/recipe_code/mamba_install.dockerfile

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
FROM jupyter/base-notebook
2+
3+
RUN mamba install --yes 'flake8' && \
4+
mamba clean --all -f -y && \
5+
fix-permissions "${CONDA_DIR}" && \
6+
fix-permissions "/home/${NB_USER}"
7+
8+
# Install from the requirements.txt file
9+
COPY --chown=${NB_UID}:${NB_GID} requirements.txt /tmp/
10+
RUN mamba install --yes --file /tmp/requirements.txt && \
11+
mamba clean --all -f -y && \
12+
fix-permissions "${CONDA_DIR}" && \
13+
fix-permissions "/home/${NB_USER}"

‎docs/using/recipe_code/manpage_install.dockerfile

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
FROM jupyter/base-notebook
2+
3+
# Fix: https://github.com/hadolint/hadolint/wiki/DL4006
4+
# Fix: https://github.com/koalaman/shellcheck/wiki/SC3014
5+
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
6+
7+
USER root
8+
9+
# `/etc/dpkg/dpkg.cfg.d/excludes` contains several `path-exclude`s, including man pages
10+
# Remove it, then install man, install docs
11+
RUN rm /etc/dpkg/dpkg.cfg.d/excludes && \
12+
apt-get update --yes && \
13+
dpkg -l | grep ^ii | cut -d' ' -f3 | xargs apt-get install --yes --no-install-recommends --reinstall man && \
14+
apt-get clean && rm -rf /var/lib/apt/lists/*
15+
16+
USER ${NB_UID}

‎docs/using/recipe_code/microsoft_odbc.dockerfile

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
FROM jupyter/base-notebook
2+
3+
# Fix: https://github.com/hadolint/hadolint/wiki/DL4006
4+
# Fix: https://github.com/koalaman/shellcheck/wiki/SC3014
5+
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
6+
7+
USER root
8+
9+
ENV MSSQL_DRIVER "ODBC Driver 18 for SQL Server"
10+
ENV PATH="/opt/mssql-tools18/bin:${PATH}"
11+
12+
RUN apt-get update --yes && \
13+
apt-get install --yes --no-install-recommends curl gnupg2 lsb-release && \
14+
curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - && \
15+
curl "https://packages.microsoft.com/config/ubuntu/$(lsb_release -rs)/prod.list" > /etc/apt/sources.list.d/mssql-release.list && \
16+
apt-get update --yes && \
17+
ACCEPT_EULA=Y apt-get install --yes --no-install-recommends msodbcsql18 && \
18+
# optional: for bcp and sqlcmd
19+
ACCEPT_EULA=Y apt-get install --yes --no-install-recommends mssql-tools18 && \
20+
# optional: for unixODBC development headers
21+
apt-get install --yes --no-install-recommends unixodbc-dev && \
22+
apt-get clean && rm -rf /var/lib/apt/lists/*
23+
24+
# Switch back to jovyan to avoid accidental container runs as root
25+
USER ${NB_UID}
26+
27+
RUN mamba install --yes 'pyodbc' && \
28+
mamba clean --all -f -y && \
29+
fix-permissions "${CONDA_DIR}" && \
30+
fix-permissions "/home/${NB_USER}"

‎docs/using/recipe_code/pip_install.dockerfile

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
FROM jupyter/base-notebook
2+
3+
# Install in the default python3 environment
4+
RUN pip install --no-cache-dir 'flake8' && \
5+
fix-permissions "${CONDA_DIR}" && \
6+
fix-permissions "/home/${NB_USER}"
7+
8+
# Install from the requirements.txt file
9+
COPY --chown=${NB_UID}:${NB_GID} requirements.txt /tmp/
10+
RUN pip install --no-cache-dir --requirement /tmp/requirements.txt && \
11+
fix-permissions "${CONDA_DIR}" && \
12+
fix-permissions "/home/${NB_USER}"

‎docs/using/recipe_code/rise_jupyterlab.dockerfile

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
FROM jupyter/base-notebook
2+
3+
RUN mamba install --yes 'jupyterlab_rise' && \
4+
mamba clean --all -f -y && \
5+
fix-permissions "${CONDA_DIR}" && \
6+
fix-permissions "/home/${NB_USER}"

‎docs/using/recipe_code/xgboost.dockerfile

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
FROM jupyter/base-notebook
2+
3+
RUN mamba install --yes 'py-xgboost' && \
4+
mamba clean --all -f -y && \
5+
fix-permissions "${CONDA_DIR}" && \
6+
fix-permissions "/home/${NB_USER}"

‎docs/using/recipes.md

Lines changed: 16 additions & 99 deletions
Original file line numberDiff line numberDiff line change
@@ -28,37 +28,14 @@ See [Docker security documentation](https://docs.docker.com/engine/security/user
2828
Create a new Dockerfile like the one shown below.
2929
To use a requirements.txt file, first, create your `requirements.txt` file with the listing of packages desired.
3030

31-
```dockerfile
32-
FROM jupyter/base-notebook
33-
34-
RUN mamba install --yes 'flake8' && \
35-
mamba clean --all -f -y && \
36-
fix-permissions "${CONDA_DIR}" && \
37-
fix-permissions "/home/${NB_USER}"
38-
39-
# Install from the requirements.txt file
40-
COPY --chown=${NB_UID}:${NB_GID} requirements.txt /tmp/
41-
RUN mamba install --yes --file /tmp/requirements.txt && \
42-
mamba clean --all -f -y && \
43-
fix-permissions "${CONDA_DIR}" && \
44-
fix-permissions "/home/${NB_USER}"
31+
```{literalinclude} recipe_code/mamba_install.dockerfile
32+
:language: docker
4533
```
4634

4735
`pip` usage is similar:
4836

49-
```dockerfile
50-
FROM jupyter/base-notebook
51-
52-
# Install in the default python3 environment
53-
RUN pip install --no-cache-dir 'flake8' && \
54-
fix-permissions "${CONDA_DIR}" && \
55-
fix-permissions "/home/${NB_USER}"
56-
57-
# Install from the requirements.txt file
58-
COPY --chown=${NB_UID}:${NB_GID} requirements.txt /tmp/
59-
RUN pip install --no-cache-dir --requirement /tmp/requirements.txt && \
60-
fix-permissions "${CONDA_DIR}" && \
61-
fix-permissions "/home/${NB_USER}"
37+
```{literalinclude} recipe_code/pip_install.dockerfile
38+
:language: docker
6239
```
6340

6441
Then build a new image.
@@ -108,17 +85,8 @@ RUN "${CONDA_DIR}/envs/${conda_env}/bin/python" -m ipykernel install --user --na
10885
[Dask JupyterLab Extension](https://github.com/dask/dask-labextension) provides a JupyterLab extension to manage Dask clusters, as well as embed Dask's dashboard plots directly into JupyterLab panes.
10986
Create the Dockerfile as:
11087

111-
```dockerfile
112-
FROM jupyter/base-notebook
113-
114-
# Install the Dask dashboard
115-
RUN mamba install --yes 'dask-labextension' && \
116-
mamba clean --all -f -y && \
117-
fix-permissions "${CONDA_DIR}" && \
118-
fix-permissions "/home/${NB_USER}"
119-
120-
# Dask Scheduler port
121-
EXPOSE 8787
88+
```{literalinclude} recipe_code/dask_jupyterlab.dockerfile
89+
:language: docker
12290
```
12391

12492
And build the image as:
@@ -153,24 +121,14 @@ We're providing the recipe to install JupyterLab extension.
153121
You can find the original Jupyter Notebook extenstion [here](https://github.com/damianavila/RISE)
154122
```
155123

156-
```dockerfile
157-
FROM jupyter/base-notebook
158-
159-
RUN mamba install --yes 'jupyterlab_rise' && \
160-
mamba clean --all -f -y && \
161-
fix-permissions "${CONDA_DIR}" && \
162-
fix-permissions "/home/${NB_USER}"
124+
```{literalinclude} recipe_code/rise_jupyterlab.dockerfile
125+
:language: docker
163126
```
164127

165128
## xgboost
166129

167-
```dockerfile
168-
FROM jupyter/base-notebook
169-
170-
RUN mamba install --yes 'py-xgboost' && \
171-
mamba clean --all -f -y && \
172-
fix-permissions "${CONDA_DIR}" && \
173-
fix-permissions "/home/${NB_USER}"
130+
```{literalinclude} recipe_code/xgboost.dockerfile
131+
:language: docker
174132
```
175133

176134
## Running behind an nginx proxy
@@ -202,19 +160,8 @@ Ref: <https://github.com/jupyter/docker-stacks/issues/199>
202160
Most containers, including our Ubuntu base image, ship without manpages installed to save space.
203161
You can use the following Dockerfile to inherit from one of our images to enable manpages:
204162

205-
```dockerfile
206-
FROM jupyter/base-notebook
207-
208-
USER root
209-
210-
# `/etc/dpkg/dpkg.cfg.d/excludes` contains several `path-exclude`s, including man pages
211-
# Remove it, then install man, install docs
212-
RUN rm /etc/dpkg/dpkg.cfg.d/excludes && \
213-
apt-get update --yes && \
214-
dpkg -l | grep ^ii | cut -d' ' -f3 | xargs apt-get install --yes --no-install-recommends --reinstall man && \
215-
apt-get clean && rm -rf /var/lib/apt/lists/*
216-
217-
USER ${NB_UID}
163+
```{literalinclude} recipe_code/manpage_install.dockerfile
164+
:language: docker
218165
```
219166

220167
Adding the documentation on top of the existing image wastes a lot of space
@@ -254,13 +201,8 @@ Credit: [Justin Tyberg](https://github.com/jtyberg), [quanghoc](https://github.c
254201
To use a specific version of JupyterHub, the version of `jupyterhub` in your image should match the
255202
version in the Hub itself.
256203

257-
```dockerfile
258-
FROM jupyter/base-notebook
259-
260-
RUN mamba install --yes 'jupyterhub==4.0.1' && \
261-
mamba clean --all -f -y && \
262-
fix-permissions "${CONDA_DIR}" && \
263-
fix-permissions "/home/${NB_USER}"
204+
```{literalinclude} recipe_code/jupyterhub_version.dockerfile
205+
:language: docker
264206
```
265207

266208
## Spark
@@ -555,33 +497,8 @@ RUN ijsinstall
555497

556498
The following recipe demonstrates how to add functionality to read from and write to an instance of Microsoft SQL server in your notebook.
557499

558-
```dockerfile
559-
FROM jupyter/base-notebook
560-
561-
USER root
562-
563-
ENV MSSQL_DRIVER "ODBC Driver 18 for SQL Server"
564-
ENV PATH="/opt/mssql-tools18/bin:${PATH}"
565-
566-
RUN apt-get update --yes && \
567-
apt-get install --yes --no-install-recommends curl gnupg2 lsb-release && \
568-
curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - && \
569-
curl https://packages.microsoft.com/config/ubuntu/$(lsb_release -rs)/prod.list > /etc/apt/sources.list.d/mssql-release.list && \
570-
apt-get update --yes && \
571-
ACCEPT_EULA=Y apt-get install --yes --no-install-recommends msodbcsql18 && \
572-
# optional: for bcp and sqlcmd
573-
ACCEPT_EULA=Y apt-get install --yes --no-install-recommends mssql-tools18 && \
574-
# optional: for unixODBC development headers
575-
apt-get install -y unixodbc-dev && \
576-
apt-get clean && rm -rf /var/lib/apt/lists/*
577-
578-
# Switch back to jovyan to avoid accidental container runs as root
579-
USER ${NB_UID}
580-
581-
RUN mamba install --yes 'pyodbc' && \
582-
mamba clean --all -f -y && \
583-
fix-permissions "${CONDA_DIR}" && \
584-
fix-permissions "/home/${NB_USER}"
500+
```{literalinclude} recipe_code/microsoft_odbc.dockerfile
501+
:language: docker
585502
```
586503

587504
You can now use `pyodbc` and `sqlalchemy` to interact with the database.

‎tensorflow-notebook/Dockerfile

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,6 @@ LABEL maintainer="Jupyter Project <jupyter@googlegroups.com>"
1111
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
1212

1313
# Install Tensorflow with pip
14-
# hadolint ignore=DL3013
1514
RUN pip install --no-cache-dir tensorflow && \
1615
fix-permissions "${CONDA_DIR}" && \
1716
fix-permissions "/home/${NB_USER}"

0 commit comments

Comments
 (0)