[RFC] Proposal for supporting FIPS 140-2 enforced mode

[terryquigleysas]

Is your feature request related to a problem? Please describe.
Feature Request #1497

Describe the solution you'd like

Problem Statement

We would like to contribute to OpenSearch to support running in FIPS-140-2 compliant mode. We propose delivering this in several phases, as discussed in the feature request above, starting with core changes and aiming towards a desired state of providing configurable options.

This RFC is to ensure our approach would be seen as a feasible and acceptable contribution.

Phases

Phase 1: Remove hardcoded Bouncy Castle references

Security plugin

Update code, retaining current functionality

• Proposed libraries: Bouncy Castle FIPS, Password4j, rfksystems Blake2b
  • Alternatively contribute to Password4j to expose Blake2b functionality and reduce the number of libraries brought in
• Security policy changes
  • Complicated by the plugin structure
• NB OpenSearch must still work for rolling upgrades

Performance Analyzer (potentially)

• This codebase is separate from OpenSearch Security and may also lead us to have to make changes to OpenSearch core
• Who do we liaise with?

Unknown unknowns (e.g. behavior of other plugins, scripts etc.)

• Emphasis on not inadvertently breaking anything

Phase 2: Introduce FIPS-compliant alternatives as default for:

Bcrypt password hashing

• PBKDF2

Blake2b for masking

• e.g. SHA3

Certificate handling (potentially)

Cipher lists (potentially)

Any additional security policy changes

Add FIPS mode configuration flag

• This may lead us to have to make changes to OpenSearch core

Phase 3: Testing and rework

By now we will be carrying out extensive testing and verification and expect that additional requirements may arise.

Additional work for any issues found in our testing

Extend unit tests

Extend integration tests

Phase 4: Configurability

Additional configuration options

• Configure additional security providers
• Configure hashing algorithms
• Validation

Contingency for unknown unknowns

Phase 5: Documentation

All required configuration options and settings

JDK 11 requirement

Limitations

Not in scope

Changing an existing cluster from non-FIPS to FIPS compliant

Dashboards, Data Prepper etc. - our focus is on server only

Any, as yet unknown, OpenSearch plugins that require extensive work for FIPS-compliance

These could be actioned by the wider community

Help Required

We have accessed and used:

• YouTube videos on developing and contributing
• Documentation and GitHub pages
• Blog items
• Slack

We expect we will need some additional help with:

• Processes
  • Proposal example
  • Creation of project stories / issues / epics / labels?
  • Backporting and releasing
  • Documentation
  • Tests
    • How do we ensure we don't break something unexpectedly?
    • Benchmarking
• Wider impact analysis
• Anything else we haven't thought of

[stephen-crawford]

[Triage] Hey @terryquigleysas, just checking in. I know you were hoping to sync up in a triaging meeting. Could you suggest a date you may be able to attend and we can make sure to prepare a slot to discuss this?

[terryquigleysas]

@scrawfor99 Sounds good. I can catch up with you on Slack to confirm.

[peternied]

Thanks for bringing up this proposal @terryquigleysas - overall the plan seems solid, but I think the details are where this project will hinge have to do with 1) making iterative improvements for updates, 2) the security approval process and 3) the project management of this release.

1. Above you've called out that there is Proposed libraries: Bouncy Castle FIPS, Password4j, rfksystems Blake2b, this seems like it might be breakable into 3 different parts - or 3 separate pull requests. I think starting with figuring out how to sub in a different Bouncy Castle implementation would be a very manageable change that builds towards the ultimate goal. I think there might be many small parts that are composed up into the ultimate objective to have a fips switch that you can turn on,

2. The maintainers of security can help with this - before we get to the end state including this feature with the generally available releases there will need to be a more in-depth review. After an minimal viable feature is available we can see about more detailed analysis.

3. Lastly, I've got notions about project planning and I might suggestion looking at the structure of one of the other [Meta] issues in the project, for a way to pivot the project that would help determine viability and then iterate through maturity. Starting with design / proof of concepts, then implementation, then integration, release criteria, and follow up improvements.

Let us know how I can help, and I can't wait to see some pull requests. Feel free to @mention me throughout the project.

[stephen-crawford]

@terryquigleysas are you still interested in pursuing this change? Thank you

[stephen-crawford]

[Triage] Going to close this. Please reopen if further work continues on this effort.

[ihendry2]

Hi we (@terryquigleysas and our team) are still interested in this change and are actively pursuing it.
At present locally we are at phase 2 of the above plan and are having issues at the integration stage.
We are mindful of your suggestions and are currently trying to work within those parameters as well.

We have also been held up with other various issues.

As such can you re-open this RFC and we will update it when we have more detail.

[davidlago]

Thanks @ihendry2. I did not see any action on this issue... where is the work on that phase 1/2 happening? Could you please link the artifacts (issues/PRs) here so that we can get the right level of engagement from the maintainer team? Thanks!

[ihendry2]

At present the changes are being worked on locally. When we have something viable we will update asap.

[terryquigleysas]

@scrawfor99 @davidlago A couple of major, unexpected events have got in the way of my looking at this among other things. I expect to pick it up in the new year and hopefully others can be brought on board.
One question in the meantime - are there still issues where settings added to the policy file can be ignored? We are seeing this as we bring in new libraries against the 2.11 code? (For example #3213)

[stephen-crawford]

Hi @terryquigleysas, the issue you linked was specifically a problem with the way that Bouncy Castle was being used. Since it was applied twice one setting would take priority over the other.