feat: support discrete environment

[Gaiejj]

Description

This pull request is aimed at supporting environments with discrete action spaces and observation spaces. It has been implemented in the Taxi-v3 and CartPole-v1 environments in Gymnasium. Relevant documents, code checks, and standards have been updated. Currently, it supports the following on-policy algorithms:

['NaturalPG', 'PolicyGradient', 'PPO', 'TRPO', 'RCPO', 'PDO', 'PPOLag', 'TRPOLag', 'OnCRPO', 'P3O', 'IPO', 'CPPOPID', 'TRPOPID', 'CPO', 'PCPO',]

to run in discrete environments.
The performance curve below shows the correctness of our implementation.

Motivation and Context

resolve #273 #283

Types of changes

What types of changes does your code introduce? Put an x in all the boxes that apply:

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds core functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

• I have read the CONTRIBUTION guide. (required)
• My change requires a change to the documentation.
• I have updated the tests accordingly. (required for a bug fix or a new feature)
• I have updated the documentation accordingly.
• I have reformatted the code using make format. (required)
• I have checked the code using make lint. (required)
• I have ensured make test pass. (required)

[codecov]

Codecov Report

Attention: Patch coverage is 90.68323% with 15 lines in your changes missing coverage. Please review.

Project coverage is 96.70%. Comparing base (d55958a) to head (e68bf5c).
Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
omnisafe/envs/discrete_env.py	85.96%	8 Missing ⚠️
omnisafe/models/base.py	63.64%	4 Missing ⚠️
omnisafe/evaluator.py	83.33%	2 Missing ⚠️
omnisafe/utils/config.py	87.50%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #286      +/-   ##
==========================================
- Coverage   96.84%   96.70%   -0.14%     
==========================================
  Files         138      140       +2     
  Lines        7000     7130     +130     
==========================================
+ Hits         6779     6895     +116     
- Misses        221      235      +14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[dvavd]

Hi! I’m currently working on a safe RL project with a discrete action environment. I came across this PR and it looks like exactly what I need. I noticed it’s been open for a while - is there any plan to merge it, or is something blocking progress?