Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 78 additions & 21 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,61 +1,118 @@
Object storage for Open edX with `MinIO <https://www.minio.io/>`_
=================================================================

This is a plugin for `Tutor <https://docs.tutor.edly.io>`_ that provides S3-like object storage for Open edX platforms. It's S3, but without the dependency on AWS. This is achieved thanks to `MinIO <https://www.minio.io/>`_, an open source project that provides object storage with an API compatible with S3.

In particular, this plugin is essential for `Kubernetes deployment <https://docs.tutor.edly.io/k8s.html>`_.
Object storage for Open edX with `Garage <https://garagehq.deuxfleurs.fr/>`_
==============================================================================

.. note::

**MinIO → Garage migration (breaking change)**

This plugin has replaced `MinIO <https://www.minio.io/>`_ with
`Garage <https://garagehq.deuxfleurs.fr/>`_ as the S3-compatible storage
backend. Open edX-side configuration (``MINIO_HOST``, bucket names,
``OPENEDX_AWS_SECRET_ACCESS_KEY``) is preserved; existing variable names
keep the ``MINIO_`` prefix for backward compatibility.

New / changed defaults:

- ``MINIO_DOCKER_IMAGE`` now points to ``docker.io/dxflrs/garage:v2.3.0``
(instead of the deprecated MinIO image).
- ``MINIO_WEBUI_DOCKER_IMAGE`` (new) — points to
``docker.io/khairul169/garage-webui:1.1.0``; replaces the legacy MinIO
console.
- ``MINIO_JOB_DOCKER_IMAGE`` (new) — locally built alpine image bundling
the ``garage`` CLI for cluster initialization.
- ``MINIO_RPC_SECRET`` (new, auto-generated) — Garage cluster shared
secret. Hashed to 32 bytes hex via the ``to_hex`` filter at render time.
- ``OPENEDX_AWS_ACCESS_KEY`` default changed from ``openedx`` to
``openedxs3`` because Garage requires key IDs to be at least 8
characters. **Existing installs that already have
``OPENEDX_AWS_ACCESS_KEY: openedx`` saved in ``config.yml`` must clear
the line so the new default takes effect**::

tutor config save --unset OPENEDX_AWS_ACCESS_KEY
tutor config save

This is a plugin for `Tutor <https://docs.tutor.edly.io>`_ that provides
S3-like object storage for Open edX platforms. It's S3, but without the
dependency on AWS. This is achieved thanks to
`Garage <https://garagehq.deuxfleurs.fr/>`_, an open-source distributed
object storage system with an S3-compatible API.

In particular, this plugin is essential for
`Kubernetes deployment <https://docs.tutor.edly.io/k8s.html>`_.

Installation
------------

The plugin is currently bundled with the `binary releases of Tutor <https://github.com/overhangio/tutor/releases>`_. If you have installed Tutor from source, you will have to install this plugin from source, too::
The plugin is currently bundled with the
`binary releases of Tutor <https://github.com/overhangio/tutor/releases>`_.
If you have installed Tutor from source, you will have to install this plugin
from source, too::

tutor plugins install minio

Then, to enable this plugin, run::

tutor plugins enable minio

The init job image is built automatically by ``tutor dev launch`` /
``tutor local launch``. To build it on demand::

tutor images build garage-job

Configuration
-------------

- ``OPENEDX_AWS_ACCESS_KEY`` (default: ``"openedx"``)
- ``OPENEDX_AWS_ACCESS_KEY`` (default: ``"openedxs3"``)
- ``OPENEDX_AWS_SECRET_ACCESS_KEY`` (default: ``"{{ 24|random_string }}"``)
- ``MINIO_BUCKET_NAME`` (default: ``"openedx"``)
- ``MINIO_FILE_UPLOAD_BUCKET_NAME`` (default: ``"openedxuploads"``)
- ``MINIO_HOST`` (default: ``"files.{{ LMS_HOST }}"``)
- ``MINIO_CONSOLE_HOST`` (default: ``"minio.{{ LMS_HOST }}"``)
- ``MINIO_DOCKER_IMAGE`` (default: ``"docker.io/minio/minio:RELEASE.2022-05-08T23-50-31Z"``)
- ``MINIO_MC_DOCKER_IMAGE`` (default: ``"docker.io/minio/mc:RELEASE.2022-05-09T04-08-26Z"``)
- ``MINIO_GRADES_BUCKET_NAME`` (default: ``"openedxgrades"``)
- ``MINIO_DOCKER_IMAGE`` (default: ``"docker.io/dxflrs/garage:v2.3.0"``)
- ``MINIO_WEBUI_DOCKER_IMAGE`` (default: ``"docker.io/khairul169/garage-webui:1.1.0"``)
- ``MINIO_JOB_DOCKER_IMAGE`` (default: ``"openedx-garage-job:{{ MINIO_VERSION }}"``)
- ``MINIO_RPC_SECRET`` (auto-generated 64-character random string, hashed to
hex for Garage)

These values can be modified with ``tutor config save --set PARAM_NAME=VALUE`` commands.

- ``MINIO_GATEWAY`` (default: ``null``)

This feature allows your to run the MinIO server as a gateway to another object storage solution, such as `S3 <https://docs.minio.io/docs/minio-gateway-for-s3.html>`__ or `Azure <https://docs.minio.io/docs/minio-gateway-for-azure.html>`__. That way, static assets can be stored on these object storage backends without extensive changes to the Open edX runtime environment.

Note to Azure users: you will have to manually grant public access rights to the ``MINIO_BUCKET_NAME`` ("openedx") bucket.
These values can be modified with ``tutor config save --set PARAM_NAME=VALUE``
commands.

DNS records
-----------

It is assumed that the ``MINIO_HOST`` DNS record points to your server. When running MinIO on your laptop, the MinIO Web UI will be available at http://minio.local.openedx.io. In development mode, the MinIO interface will be available at http://minio.local.openedx.io:9001.
It is assumed that the ``MINIO_HOST`` DNS record points to your server. When
running Garage on your laptop, the S3 endpoint will be available at
http://files.local.openedx.io. In development mode, the S3 API is also
exposed on http://127.0.0.1:9000.

Web UI
------

The MinIO web UI can be accessed at http://<MINIO_HOST>. The credentials for accessing the UI can be obtained with::
The Garage web UI is accessible at:

- **Local/production mode**: ``http://<MINIO_CONSOLE_HOST>`` (proxied through
Caddy)
- **Development mode**: ``http://<MINIO_CONSOLE_HOST>:3909``

It auto-reads the admin token from the mounted ``/etc/garage.toml``, so no
explicit login is required. Use the access key + secret to manage objects::

tutor config printvalue OPENEDX_AWS_ACCESS_KEY
tutor config printvalue OPENEDX_AWS_SECRET_ACCESS_KEY

Troubleshooting
---------------

This Tutor plugin is maintained by Abdul Rehman from `Edly <https://edly.io>`__. Community support is available from the official `Open edX forum <https://discuss.openedx.org>`__. Do you need help with this plugin? See the `troubleshooting <https://docs.tutor.edly.io/troubleshooting.html>`__ section from the Tutor documentation.
This Tutor plugin is maintained by Abdul Rehman from `Edly <https://edly.io>`__.
Community support is available from the official
`Open edX forum <https://discuss.openedx.org>`__. Do you need help with this
plugin? See the
`troubleshooting <https://docs.tutor.edly.io/troubleshooting.html>`__ section
from the Tutor documentation.

License
-------

This work is licensed under the terms of the `GNU Affero General Public License (AGPL) <https://github.com/overhangio/tutor-minio/blob/release/LICENSE.txt>`_.
This work is licensed under the terms of the
`GNU Affero General Public License (AGPL) <https://github.com/overhangio/tutor-minio/blob/release/LICENSE.txt>`_.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- 💥[Feature] Replace MinIO with [Garage v2.3.0](https://garagehq.deuxfleurs.fr/) as the S3-compatible storage backend, plus the [garage-webui](https://github.com/khairul169/garage-webui) admin console. Open edX-side configuration is preserved (variable names keep the `MINIO_` prefix). New variables: `MINIO_WEBUI_DOCKER_IMAGE`, `MINIO_JOB_DOCKER_IMAGE`, `MINIO_RPC_SECRET`. The `OPENEDX_AWS_ACCESS_KEY` default changes from `openedx` to `openedxs3` to satisfy Garage's 8-character minimum key ID — existing installs must clear the saved line (`tutor config save --unset OPENEDX_AWS_ACCESS_KEY && tutor config save`) before re-launch. (By @Syed-Ali-Abbas-568)
6 changes: 3 additions & 3 deletions tutorminio/patches/caddyfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# MinIO
# Garage
{{ MINIO_HOST }}{$default_site_port} {
import proxy "minio:9000"
import proxy "garage:9000"
}
{{ MINIO_CONSOLE_HOST }}{$default_site_port} {
import proxy "minio:9001"
import proxy "garage-webui:3909"
}
111 changes: 89 additions & 22 deletions tutorminio/patches/k8s-deployments
Original file line number Diff line number Diff line change
@@ -1,45 +1,112 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: garage-config
labels:
app.kubernetes.io/name: garage
data:
garage.toml: |
metadata_dir = "/var/lib/garage/meta"
data_dir = "/var/lib/garage/data"
db_engine = "lmdb"
replication_factor = 1

rpc_bind_addr = "[::]:3901"
rpc_secret = "{{ MINIO_RPC_SECRET | to_hex }}"

[s3_api]
s3_region = "us-east-1"
api_bind_addr = "[::]:9000"
root_domain = ".s3.garage"

[s3_web]
bind_addr = "[::]:3902"
root_domain = ".web.garage"

[admin]
api_bind_addr = "[::]:3903"
admin_token = "{{ MINIO_RPC_SECRET | to_hex }}"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: minio
name: garage
labels:
app.kubernetes.io/name: minio
app.kubernetes.io/name: garage
spec:
selector:
matchLabels:
app.kubernetes.io/name: minio
app.kubernetes.io/name: garage
strategy:
type: Recreate
template:
metadata:
labels:
app.kubernetes.io/name: minio
app.kubernetes.io/name: garage
spec:
containers:
- name: minio
- name: garage
image: {{ MINIO_DOCKER_IMAGE }}
{% if MINIO_GATEWAY %}
args: ["gateway", "{{ MINIO_GATEWAY }}", "--address", ":9000", "--console-address", ":9001"]
{% else %}
args: ["server", "--address", ":9000", "--console-address", ":9001", "/data"]
{% endif %}
env:
- name: MINIO_ROOT_USER
value: "{{ OPENEDX_AWS_ACCESS_KEY }}"
- name: MINIO_ROOT_PASSWORD
value: "{{ OPENEDX_AWS_SECRET_ACCESS_KEY }}"
ports:
- containerPort: 9000
- containerPort: 9001
{% if not MINIO_GATEWAY %}
name: s3-api
- containerPort: 3901
name: rpc
- containerPort: 3903
name: admin-api
volumeMounts:
- mountPath: /data
- mountPath: /etc/garage.toml
name: config
subPath: garage.toml
- mountPath: /var/lib/garage/meta
name: meta
- mountPath: /var/lib/garage/data
name: data
{% endif %}
{% if not MINIO_GATEWAY %}
volumes:
- name: config
configMap:
name: garage-config
- name: meta
persistentVolumeClaim:
claimName: garage-meta
- name: data
persistentVolumeClaim:
claimName: minio
{% endif %}
claimName: garage-data
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: garage-webui
labels:
app.kubernetes.io/name: garage-webui
spec:
selector:
matchLabels:
app.kubernetes.io/name: garage-webui
template:
metadata:
labels:
app.kubernetes.io/name: garage-webui
spec:
containers:
- name: garage-webui
image: {{ MINIO_WEBUI_DOCKER_IMAGE }}
ports:
- containerPort: 3909
name: http
env:
- name: API_BASE_URL
value: "http://garage:3903"
- name: S3_ENDPOINT_URL
value: "http://garage:9000"
- name: CONFIG_PATH
value: "/etc/garage.toml"
volumeMounts:
- mountPath: /etc/garage.toml
name: config
subPath: garage.toml
volumes:
- name: config
configMap:
name: garage-config
12 changes: 5 additions & 7 deletions tutorminio/patches/k8s-jobs
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,17 @@
apiVersion: batch/v1
kind: Job
metadata:
name: minio-job
name: garage-job
labels:
app.kubernetes.io/component: job
spec:
template:
spec:
restartPolicy: Never
containers:
- name: minio
image: {{ MINIO_MC_DOCKER_IMAGE }}
- name: garage
image: {{ MINIO_JOB_DOCKER_IMAGE }}
command: []
env:
- name: MINIO_ROOT_USER
value: "{{ OPENEDX_AWS_ACCESS_KEY }}"
- name: MINIO_ROOT_PASSWORD
value: "{{ OPENEDX_AWS_SECRET_ACCESS_KEY }}"
- name: GARAGE_RPC_SECRET
value: "{{ MINIO_RPC_SECRET | to_hex }}"
26 changes: 21 additions & 5 deletions tutorminio/patches/k8s-services
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,31 @@
apiVersion: v1
kind: Service
metadata:
name: minio
name: garage
spec:
type: NodePort
ports:
- port: 9000
protocol: TCP
name: minio-server
- port: 9001
name: s3-api
- port: 3901
protocol: TCP
name: minio-console
name: rpc
- port: 3903
protocol: TCP
name: admin-api
selector:
app.kubernetes.io/name: garage
---
apiVersion: v1
kind: Service
metadata:
name: garage-webui
spec:
type: NodePort
ports:
- port: 3909
protocol: TCP
name: http
selector:
app.kubernetes.io/name: minio
app.kubernetes.io/name: garage-webui
20 changes: 16 additions & 4 deletions tutorminio/patches/k8s-volumes
Original file line number Diff line number Diff line change
@@ -1,16 +1,28 @@
{% if not MINIO_GATEWAY %}
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: minio
name: garage-meta
labels:
app.kubernetes.io/component: volume
app.kubernetes.io/name: minio
app.kubernetes.io/name: garage
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: garage-data
labels:
app.kubernetes.io/component: volume
app.kubernetes.io/name: garage
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
{% endif %}
2 changes: 1 addition & 1 deletion tutorminio/patches/local-docker-compose-cms-dependencies
Original file line number Diff line number Diff line change
@@ -1 +1 @@
- minio
- garage
Loading
Loading