create openedx envs common settings #36941

wgu-taylor-payne · 2025-06-23T15:45:05Z

Description

Pull settings explicitly shared by the LMS and CMS into a common settings module at openedx/envs/common.py that lms/envs/common.py and cms/envs/common.py inherit from. For now, only settings explicitly shared settings (via import in cms/envs/common.py) were targeted to move up into the new shared module.

This is one of the initial steps in the effort to simplify Django settings in the edx-platform (see the related ADR: 0022-settings-simplification).

Documented settings in openedx/envs/common.py will be included on the settings reference page.

Supporting information

Resolves #36889.

Testing instructions

`diff_settings.py`

I've used the diff_settings.sh script as a foundation to create a diff_settings.py script that can be found here (documentation here). I've used this script to test the rendered settings for this branch against the rendered settings in the master branch.

Modification to `cms/envs/production.py` required

To get all runs of dump_settings to work (specifically running the command with DJANGO_SETTINGS_MODULE=cms.envs.production and CMS_CFG=cms/envs/mock.yml) I needed to make a small patch to cms/envs/production.py in both master and my branch:

-CSRF_TRUSTED_ORIGINS = _YAML_TOKENS.get("CSRF_TRUSTED_ORIGINS", [])
+CSRF_TRUSTED_ORIGINS = _YAML_TOKENS.get('CSRF_TRUSTED_ORIGINS_WITH_SCHEME', [])

Without this, the command would fail and output this error:

As of Django 4.0, the values in the CSRF_TRUSTED_ORIGINS setting must start with a scheme (usually http:// or https://) but found .localhost. See the release notes for details.

Results from running `diff_settings.py`

Running python diff_settings.py master tpayne/create-openedx-envs-common-settings in a tutor dev environment, I get the following results:

Starting Django settings dump comparison...
Comparing master -> tpayne/create-openedx-envs-common-settings
Output: main_settings_dump/ and feature_settings_dump/

Cleaning existing directory: main_settings_dump
Cleaning existing directory: feature_settings_dump
=== Processing branch: master ===
Processing configurations  [####################################]  100%
Completed processing branch: master

=== Processing branch: tpayne/create-openedx-envs-common-settings ===
Processing configurations  [####################################]  100%
Completed processing branch: tpayne/create-openedx-envs-common-settings

=== Running diff comparison ===
diff main_settings_dump/cms.envs.production__cms_envs_mock.json feature_settings_dump/cms.envs.production__cms_envs_mock.json
4360c4360
<             "module": "lms.envs.common",
---
>             "module": "openedx.envs.common",
4889c4889
<                 "stream": "<colorama.ansitowin32.StreamWrapper object at 0xffff86001fd0>"
---
>                 "stream": "<colorama.ansitowin32.StreamWrapper object at 0xffff7e80bf90>"
5847a5848
>     "USAGE_ID_PATTERN": "(?P<usage_id>(?:i4x://?[^/]+/[^/]+/[^/]+/[^@]+(?:@[^/]+)?)|(?:[^/]+))",
diff main_settings_dump/cms.envs.tutor.development__openedx_config_cms.json feature_settings_dump/cms.envs.tutor.development__openedx_config_cms.json
2084c2084
<             "module": "lms.envs.common",
---
>             "module": "openedx.envs.common",
2607c2607
<                 "stream": "<colorama.ansitowin32.StreamWrapper object at 0xffffa6389fd0>"
---
>                 "stream": "<colorama.ansitowin32.StreamWrapper object at 0xffff828d9fd0>"
3477a3478
>     "USAGE_ID_PATTERN": "(?P<usage_id>(?:i4x://?[^/]+/[^/]+/[^/]+/[^@]+(?:@[^/]+)?)|(?:[^/]+))",
diff main_settings_dump/cms.envs.tutor.production__openedx_config_cms.json feature_settings_dump/cms.envs.tutor.production__openedx_config_cms.json
2048c2048
<             "module": "lms.envs.common",
---
>             "module": "openedx.envs.common",
2571c2571
<                 "stream": "<colorama.ansitowin32.StreamWrapper object at 0xffff7f8f9fd0>"
---
>                 "stream": "<colorama.ansitowin32.StreamWrapper object at 0xffffaa8f9fd0>"
3440a3441
>     "USAGE_ID_PATTERN": "(?P<usage_id>(?:i4x://?[^/]+/[^/]+/[^/]+/[^@]+(?:@[^/]+)?)|(?:[^/]+))",
diff main_settings_dump/lms.envs.production__lms_envs_minimal.json feature_settings_dump/lms.envs.production__lms_envs_minimal.json
2365c2365
<             "module": "lms.envs.common",
---
>             "module": "openedx.envs.common",
2863c2863
<                 "stream": "<colorama.ansitowin32.StreamWrapper object at 0xffffa753af90>"
---
>                 "stream": "<colorama.ansitowin32.StreamWrapper object at 0xffff8c23dc90>"
diff main_settings_dump/lms.envs.production__lms_envs_mock.json feature_settings_dump/lms.envs.production__lms_envs_mock.json
5856c5856
<             "module": "lms.envs.common",
---
>             "module": "openedx.envs.common",
6365c6365
<                 "stream": "<colorama.ansitowin32.StreamWrapper object at 0xffff91eadc50>"
---
>                 "stream": "<colorama.ansitowin32.StreamWrapper object at 0xffffbcef3910>"
diff main_settings_dump/lms.envs.tutor.development__openedx_config_lms.json feature_settings_dump/lms.envs.tutor.development__openedx_config_lms.json
2689c2689
<             "module": "lms.envs.common",
---
>             "module": "openedx.envs.common",
3381c3381
<                 "stream": "<colorama.ansitowin32.StreamWrapper object at 0xffffba7d9e90>"
---
>                 "stream": "<colorama.ansitowin32.StreamWrapper object at 0xffff9e9cdc50>"
diff main_settings_dump/lms.envs.tutor.production__openedx_config_lms.json feature_settings_dump/lms.envs.tutor.production__openedx_config_lms.json
2414c2414
<             "module": "lms.envs.common",
---
>             "module": "openedx.envs.common",
2914c2914
<                 "stream": "<colorama.ansitowin32.StreamWrapper object at 0xffffb61cafd0>"
---
>                 "stream": "<colorama.ansitowin32.StreamWrapper object at 0xffffa4a3ed50>"

Differences found between branches (shown above)

The "module" lines are describing where the lambda for the JWT_AUTH setting JWT_PAYLOAD_GET_USERNAME_HANDLER is being defined.

Also, I decided to bring USAGE_ID_PATTERN, which historically hasn't been imported in the CMS settings, into openedx/envs/common.py along with USAGE_KEY_PATTERN and ASSET_KEY_PATTERN, which have been imported into the CMS settings. These settings are similar and I thought it would make sense to declare them in the same place.

I also made a modification to HEARTBEAT_CELERY_ROUTING_KEY in cms/envs/common.py where it will contain 'cms' rather than 'lms' in the value, if SERVICE_VARIANT is not set (I don't know if this would be a common scenario or not). To test the difference with this setting I had to modify manage.py to not set SERVICE_VARIANT if unset. I also had to modify diff_settings.py to unset the SERVICE_VARIANT before running the dump_settings management command. With those changes in place these differences appeared:

diff main_settings_dump/cms.envs.production__cms_envs_mock.json feature_settings_dump/cms.envs.production__cms_envs_mock.json
4154c4154
<     "HEARTBEAT_CELERY_ROUTING_KEY": "edx.lms.core.high",
---
>     "HEARTBEAT_CELERY_ROUTING_KEY": "edx.cms.core.high",

diff main_settings_dump/cms.envs.tutor.development__openedx_config_cms.json feature_settings_dump/cms.envs.tutor.development__openedx_config_cms.json
1883c1883
<     "HEARTBEAT_CELERY_ROUTING_KEY": "edx.lms.core.high",
---
>     "HEARTBEAT_CELERY_ROUTING_KEY": "edx.cms.core.high",

diff main_settings_dump/cms.envs.tutor.production__openedx_config_cms.json feature_settings_dump/cms.envs.tutor.production__openedx_config_cms.json
1853c1853
<     "HEARTBEAT_CELERY_ROUTING_KEY": "edx.lms.core.high",
---
>     "HEARTBEAT_CELERY_ROUTING_KEY": "edx.cms.core.high",

openedx-webhooks · 2025-06-23T15:45:10Z

Thanks for the pull request, @wgu-taylor-payne!

This repository is currently maintained by @openedx/wg-maintenance-edx-platform.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
- This process (including the steps you'll need to take) is documented here.
If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

Dependencies

This PR must be merged before / after / at the same time as ...
Blockers

This PR is waiting for OEP-1234 to be accepted.
Timeline information

This PR must be merged by XX date because ...
Partner information

This is for a course on edx.org.
Supporting documentation
Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

The size and impact of the changes that it introduces
The need for product review
Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

wgu-taylor-payne · 2025-06-23T23:22:11Z

cms/envs/common.py

+
+from openedx.core.constants import COURSE_KEY_REGEX, COURSE_KEY_PATTERN, COURSE_ID_PATTERN
+from openedx.envs.common import *  # pylint: disable=wildcard-import
+
 from lms.envs.common import (


I didn't move these up, since it is unclear whether they belong in the CMS long term.

wgu-taylor-payne · 2025-06-24T19:57:46Z

openedx/envs/common.py

+To create section headers in this file, use the following function:
+
+```python
+def center_with_hashes(text: str, width: int = 76):
+    print(f"{f' {text} ':#^{width}}")
+```


Not sure if this should be included, but if we decide to continue marking sections in this way, it is nice to be able to create them in a standardized style.

wgu-taylor-payne · 2025-06-24T20:00:13Z

openedx/envs/common.py

+MEDIA_URL = '/media/'
+
+# Dummy secret key for dev/test
+SECRET_KEY = 'dev key'


This is a variable which wasn't explicitly imported into the cms settings from the lms settings, but it is used by such a setting (JWT_AUTH).

wgu-taylor-payne · 2025-06-24T20:05:24Z

openedx/envs/common.py

+
+DEBUG_TOOLBAR_PATCH_SETTINGS = False
+
+################ Shared Functions for Derived Configuration ################


I have decided to improve the docstrings for the two functions in this section. Also, since they are declared here and then imported into the lms and cms settings modules, I have removed the leading _ in their names which had marked them as private.

Doing this also makes them available if we use from openedx.envs.common import * to import the things defined in this file into downstream settings files.

wgu-taylor-payne · 2025-06-24T20:09:46Z

openedx/envs/common.py

+# .. setting_name: PARENTAL_CONSENT_AGE_LIMIT
+# .. setting_default: 13
+# .. setting_description: The age at which a learner no longer requires parental consent,
+#   or ``None`` if parental consent is never required.
+PARENTAL_CONSENT_AGE_LIMIT = 13
+
+############################### Registration ###############################
+
+# .. setting_name: REGISTRATION_EMAIL_PATTERNS_ALLOWED
+# .. setting_default: None
+# .. setting_description: Optional setting to restrict registration / account creation
+#   to only emails that match a regex in this list. Set to ``None`` to allow any email (default).
+REGISTRATION_EMAIL_PATTERNS_ALLOWED = None


This documentation did not exist, but I added it to make sure that such documentation declared in this module would appear in the settings reference docs page.

wgu-taylor-payne · 2025-06-24T20:11:05Z

openedx/envs/common.py

+    'small': 30
+}
+
+############################## Miscellaneous ###############################


I wasn't readily sure how to categorize the settings added here.

wgu-taylor-payne · 2025-06-24T20:16:59Z

openedx/envs/common.py

+# They are used so that URLs with deprecated-format strings still work.
+USAGE_KEY_PATTERN = r'(?P<usage_key_string>(?:i4x://?[^/]+/[^/]+/[^/]+/[^@]+(?:@[^/]+)?)|(?:[^/]+))'
+ASSET_KEY_PATTERN = r'(?P<asset_key_string>(?:/?c4x(:/)?/[^/]+/[^/]+/[^/]+/[^@]+(?:@[^/]+)?)|(?:[^/]+))'
+USAGE_ID_PATTERN = r'(?P<usage_id>(?:i4x://?[^/]+/[^/]+/[^/]+/[^@]+(?:@[^/]+)?)|(?:[^/]+))'


USAGE_KEY_PATTERN and ASSET_KEY_PATTERN were being imported into the cms settings from the lms settings, but USAGE_ID_PATTERN was not. I decided to add USAGE_ID_PATTERN here since it is closely related to the other two settings. If we don't care as much about the co-location of these settings, then we can move USAGE_ID_PATTERN back into lms/envs/common.py.

Do these really need to be a setting? I'm wondering if we can just move them to a constants file.

Good point, these do seem more like constants than configuration. As a side note, these type of URIs have been deprecated, and would only be used for backwards compatibility at this point from what I understand.

It looks like openedx/core/constants.py would be a good place to add them. It looks like the constants in this file are mostly being accessed via django.conf.settings at this time, but there are some instances where they are directly accessed from openedx.core.constants. Since they are already being accessed via settings, for this PR, I think we should add them to the constants.py module, but then still import them into the settings module.

Looking forward though, I think it ultimately makes the most sense to import constants directly from the constants module, and if a constant ever needs to be overridden by configuration, it should be moved into settings.

I've added these to openedx/core/constants.py, and then import them into openedx/envs/common.py.

docs/references/settings.rst

wgu-taylor-payne · 2025-06-24T20:29:24Z

cms/envs/common.py

@@ -1539,6 +1458,10 @@
 CELERY_BROKER_USE_SSL = False
 CELERY_EVENT_QUEUE_TTL = None

+############################## HEARTBEAT ######################################
+
+HEARTBEAT_CELERY_ROUTING_KEY = HIGH_PRIORITY_QUEUE


HEARTBEAT_CELERY_ROUTING_KEY has previously been imported from lms/envs/common.py, but when I was looking into it, that LMS setting can/will contain lms in this str in the event that SERVICE_VARIANT is not set (I'm not sure how rare this scenario would be).

These variables follow a similar pattern in both lms/envs/common.py and cms/envs/common.py. Here is how HEARTBEAT_CELERY_ROUTING_KEY gets its value in lms/envs/common.py:

SERVICE_VARIANT = os.environ.get('SERVICE_VARIANT', 'lms') # 'cms' in cms/envs/common.py CONFIG_PREFIX = SERVICE_VARIANT + "." if SERVICE_VARIANT else "" QUEUE_VARIANT = CONFIG_PREFIX.lower() HIGH_PRIORITY_QUEUE = f'edx.{QUEUE_VARIANT}core.high' HEARTBEAT_CELERY_ROUTING_KEY = HIGH_PRIORITY_QUEUE

So, cms/envs/common.py has the same pattern (except it defaults to 'cms' for the SERVICE_VARIANT), but then it imports HEARTBEAT_CELERY_ROUTING_KEY from the lms settings rather than using HIGH_PRIORITY_QUEUE as the value like in lms/envs/common.py.

Is that how we should keep it, or should the HEARTBEAT_CELERY_ROUTING_KEY value in the cms settings contain 'cms' rather than 'lms' in the key when/if the SERVICE_VARIANT is unset?

Another thing to add here is I can't find anywhere the cms uses HEARTBEAT_CELERY_ROUTING_KEY.

wgu-taylor-payne · 2025-06-24T20:36:34Z

I have left in 'DIRS': lms.envs.common.MAKO_TEMPLATE_DIRS_BASE, (see here) in cms/envs/common.py:664 because it is using a setting that is explicitly set for the LMS.

kdmccormick · 2025-06-26T14:01:50Z

Excellent work. I'm busy for the next couple days with conference preparation, so I'll see if any other CCs are available to review. If not, I'll review as soon as I'm able to.

open-craft-grove · 2025-06-26T19:00:23Z

Sandbox deployment failed 💥
Please check the settings and requirements.
Retry deployment by pushing a new commit or updating the requirements/settings in the pull request's description.
📜 Failure Logs
ℹ️ Grove Config, Tutor Config, Tutor Requirements

open-craft-grove · 2025-06-26T21:17:59Z

Sandbox deployment successful 🚀
🎓 LMS
📝 Studio
ℹ️ Grove Config, Tutor Config, Tutor Requirements

open-craft-grove · 2025-06-29T17:26:39Z

Sandbox deployment successful 🚀
🎓 LMS
📝 Studio
ℹ️ Grove Config, Tutor Config, Tutor Requirements

open-craft-grove · 2025-07-02T20:03:19Z

Sandbox deployment successful 🚀
🎓 LMS
📝 Studio
ℹ️ Grove Config, Tutor Config, Tutor Requirements

openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Jun 23, 2025

openedx-webhooks added this to Contributions Jun 23, 2025

github-project-automation bot moved this to Needs Triage in Contributions Jun 23, 2025

wgu-taylor-payne marked this pull request as draft June 23, 2025 15:50

wgu-taylor-payne self-assigned this Jun 23, 2025

wgu-taylor-payne force-pushed the tpayne/create-openedx-envs-common-settings branch from 48c86a1 to 9c610f5 Compare June 23, 2025 19:52

wgu-taylor-payne commented Jun 23, 2025

View reviewed changes

wgu-taylor-payne force-pushed the tpayne/create-openedx-envs-common-settings branch from 8b509eb to eaea6f7 Compare June 24, 2025 18:43

mphilbrick211 added the mao-onboarding Reviewing this will help onboard devs from an Axim mission-aligned organization (MAO). label Jun 24, 2025

wgu-taylor-payne commented Jun 24, 2025

View reviewed changes

wgu-taylor-payne marked this pull request as ready for review June 24, 2025 20:40

wgu-taylor-payne requested a review from a team as a code owner June 24, 2025 20:40

wgu-taylor-payne requested review from feanil and kdmccormick June 24, 2025 20:41

wgu-taylor-payne marked this pull request as draft June 25, 2025 18:08

wgu-taylor-payne marked this pull request as ready for review June 26, 2025 01:37

kdmccormick moved this from Needs Triage to Ready for Review in Contributions Jun 26, 2025

kdmccormick added the create-sandbox open-craft-grove should create a sandbox environment from this PR label Jun 26, 2025

wgu-taylor-payne added 3 commits July 2, 2025 12:18

refactor: move explicitly shared settings into new common module

7766e96

docs: add platform-wide settings section to settings reference

e61feab

refactor: import constants into platform-wide settings module

4ba8298

wgu-taylor-payne force-pushed the tpayne/create-openedx-envs-common-settings branch from eaea6f7 to 4ba8298 Compare July 2, 2025 18:19

wgu-taylor-payne requested a review from bradenmacdonald July 2, 2025 18:23


		DEBUG_TOOLBAR_PATCH_SETTINGS = False

		################ Shared Functions for Derived Configuration ################

create openedx envs common settings #36941

Are you sure you want to change the base?

create openedx envs common settings #36941

Conversation

wgu-taylor-payne commented Jun 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Supporting information

Testing instructions

diff_settings.py

Modification to cms/envs/production.py required

Results from running diff_settings.py

Uh oh!

openedx-webhooks commented Jun 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wgu-taylor-payne Jun 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wgu-taylor-payne commented Jun 24, 2025

Uh oh!

kdmccormick commented Jun 26, 2025

Uh oh!

open-craft-grove commented Jun 26, 2025

Uh oh!

open-craft-grove commented Jun 26, 2025

Uh oh!

open-craft-grove commented Jun 29, 2025

Uh oh!

open-craft-grove commented Jul 2, 2025

Uh oh!

Uh oh!

wgu-taylor-payne commented Jun 23, 2025 •

edited

Loading

`diff_settings.py`

Modification to `cms/envs/production.py` required

Results from running `diff_settings.py`

openedx-webhooks commented Jun 23, 2025 •

edited

Loading

wgu-taylor-payne Jun 24, 2025 •

edited

Loading