Skip to content

Support IMDS V2.1 #3255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 46 commits into
base: version-3
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
a211c53
Remove IMDSv1 mechanism in InstanceProfileCreds
jterapin Jun 2, 2025
7f0126d
Use ec2 metadata client to fetch credentials for instance profile
jterapin Jun 5, 2025
22c1bec
Fix handling of parsing creds
jterapin Jun 5, 2025
9a5410d
Clean up
jterapin Jun 5, 2025
bca31d5
Tidy up existing code
jterapin Jun 6, 2025
bf2fabb
Fix doc warning
jterapin Jun 6, 2025
2c5cca4
Add new config opts
jterapin Jun 6, 2025
350c6f3
Merge branch 'version-3' into imds_update
jterapin Jun 6, 2025
f018568
Minor doc fixes
jterapin Jun 6, 2025
5323f6c
IMDS 2.1 refactor
jterapin Jun 11, 2025
7527feb
Fix failing credential resolution chain specs
jterapin Jun 11, 2025
7349b61
Update shared config and its specs
jterapin Jun 11, 2025
b1de27f
Update chain specs
jterapin Jun 11, 2025
8f11f99
Update ec2 metadata specs
jterapin Jun 11, 2025
de3c2ca
Clean up ec2 metadata
jterapin Jun 11, 2025
82d5dd1
Minor update to EC2 metadata specs
jterapin Jun 11, 2025
f8a25e3
Clean up IMDS provider
jterapin Jun 11, 2025
89d42ba
Small update
jterapin Jun 11, 2025
2951add
Merge branch 'version-3' into imds_update
jterapin Jun 11, 2025
5cb5a41
Changelog draft
jterapin Jun 11, 2025
3856e44
Add back attribute
jterapin Jun 11, 2025
5f59afa
More tidyness
jterapin Jun 11, 2025
42ca582
Remove json checks
jterapin Jun 12, 2025
ff0500d
Remove redunant boolean expressions from disable ec2 metadata check
jterapin Jun 12, 2025
ba66dab
Update docs rendering for ec2 metadata
jterapin Jun 12, 2025
da697a7
Update EC2 specs
jterapin Jun 12, 2025
efce766
Remove trailing comma
jterapin Jun 12, 2025
0f9217d
Some feedback updates
jterapin Jun 12, 2025
ff4b2bd
Update outdated doc links
jterapin Jun 13, 2025
37d5cc5
Update imds provider
jterapin Jun 13, 2025
84762d1
update specs
jterapin Jun 13, 2025
d3ed775
Address feedbacks
jterapin Jun 13, 2025
e39fd9d
Minor clean up
jterapin Jun 13, 2025
f107214
Update changelog
jterapin Jun 13, 2025
4f55b42
Small clean up on chain
mullermp Jun 13, 2025
f759f7e
Add back some ec2metadata tests
mullermp Jun 13, 2025
168e077
Clean up of instance profile credentials
mullermp Jun 13, 2025
ac54816
Minor updates
jterapin Jun 13, 2025
ba05271
Add test runner
jterapin Jun 14, 2025
a4aabe7
Minor clean ups
jterapin Jun 14, 2025
3e38139
Fix ruby 2.7
mullermp Jun 14, 2025
aba9a45
WIP - test issues
mullermp Jun 14, 2025
dffaeb1
Add tests verbatum
mullermp Jun 14, 2025
e0d2224
Fix resolution tests
mullermp Jun 14, 2025
15f1b0d
socket error for extended path across all specs
mullermp Jun 14, 2025
5d37ab7
Update IMDS provider
jterapin Jun 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions gems/aws-sdk-core/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,14 @@
Unreleased Changes
------------------

* Feature - Support sourcing account ID when fetching credentials using `Aws::InstanceProfileCredentials`.

* Feature - Remove support for disabling IMDSv1 in the `InstanceProfileCredentials` provider. The following configurations has been removed: `ENV['AWS_EC2_METADATA_V1_DISABLED']`, `ec2_metadata_v1_disabled` shared config, or the `disable_imds_v1` credentials option.

* Feature - Support shared config option `disable_ec2_metadata` to disable IMDS credential fetching, in addition to the existing `ENV['AWS_EC2_METADATA_DISABLED']` option.

* Feature - Support configuration for IMDS profile name using `ENV['AWS_EC2_INSTANCE_PROFILE_NAME']`, `ec2_instance_profile_name` shared config, or the `ec2_instance_profile_name` Client configuration option.

3.225.2 (2025-06-10)
------------------

Expand Down
10 changes: 9 additions & 1 deletion gems/aws-sdk-core/lib/aws-sdk-core/credential_provider_chain.rb
Original file line number Diff line number Diff line change
Expand Up @@ -191,11 +191,19 @@ def instance_profile_credentials(options)
if ENV['AWS_CONTAINER_CREDENTIALS_RELATIVE_URI'] ||
ENV['AWS_CONTAINER_CREDENTIALS_FULL_URI']
ECSCredentials.new(options)
else
elsif !ec2_metadata_disabled(profile_name)
InstanceProfileCredentials.new(options.merge(profile: profile_name))
end
end

def ec2_metadata_disabled(profile_name)
value =
ENV['AWS_EC2_METADATA_DISABLED'] ||
Aws.shared_config.config_enabled? && Aws.shared_config.disable_ec2_metadata(profile: profile_name) ||
'false'
Aws::Util.str_2_bool(value)
end

def assume_role_with_profile(options, profile_name)
assume_opts = {
profile: profile_name,
Expand Down
132 changes: 74 additions & 58 deletions gems/aws-sdk-core/lib/aws-sdk-core/ec2_metadata.rb
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# frozen_string_literal: true

require 'time'
require 'net/http'
require 'time'

module Aws
# A client that can query version 2 of the EC2 Instance Metadata
# A client that can query version 2 of the EC2 Instance Metadata.
class EC2Metadata
# Path for PUT request for token
# @api private
METADATA_TOKEN_PATH = '/latest/api/token'.freeze
METADATA_TOKEN_PATH = '/latest/api/token'

# Raised when the PUT request is not valid. This would be thrown if
# `token_ttl` is not an Integer.
Expand All @@ -33,34 +33,33 @@ class RequestForbiddenError < RuntimeError; end
# @note Customers using containers may need to increase their hop limit
# to access IMDSv2.
# @see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html#instance-metadata-transition-to-version-2
# Use the Instance Metadata Service to access instance metadata.
#
# @param [Hash] options
# @option options [Integer] :token_ttl (21600) The session token's TTL,
# defaulting to 6 hours.
# @option options [Integer] :retries (3) The number of retries for failed
# requests.
# @option options [String] :endpoint ('http://169.254.169.254') The IMDS
# endpoint. This option has precedence over the :endpoint_mode.
# endpoint. This option has precedence over the `:endpoint_mode`.
# @option options [String] :endpoint_mode ('IPv4') The endpoint mode for
# the instance metadata service. This is either 'IPv4'
# ('http://169.254.169.254') or 'IPv6' ('http://[fd00:ec2::254]').
# the instance metadata service. This is either `'IPv4'
# ('http://169.254.169.254')` or `'IPv6' ('http://[fd00:ec2::254]')`.
# @option options [Integer] :port (80) The IMDS endpoint port.
# @option options [Integer] :http_open_timeout (1) The number of seconds to
# wait for the connection to open.
# @option options [Integer] :http_read_timeout (1) The number of seconds for
# one chunk of data to be read.
# @option options [IO] :http_debug_output An output stream for debugging. Do
# not use this in production.
# @option options [Integer,Proc] :backoff A backoff used for retryable
# @option options [Integer, Proc] :backoff A backoff used for retryable
# requests. When given an Integer, it sleeps that amount. When given a
# Proc, it is called with the current number of failed retries.
def initialize(options = {})
@token_ttl = options[:token_ttl] || 21_600
@retries = options[:retries] || 3
@backoff = backoff(options[:backoff])

endpoint_mode = options[:endpoint_mode] || 'IPv4'
@endpoint = resolve_endpoint(options[:endpoint], endpoint_mode)
@backoff = resolve_backoff(options[:backoff])
@endpoint = resolve_endpoint(options)
@port = options[:port] || 80

@http_open_timeout = options[:http_open_timeout] || 1
Expand All @@ -71,66 +70,81 @@ def initialize(options = {})
@mutex = Mutex.new
end

# @return [Integer]
attr_reader :token_ttl

# @return [Integer]
attr_reader :retries

# @return [Proc]
attr_reader :backoff

# @return [String]
attr_reader :endpoint

# @return [Integer]
attr_reader :port

# @return [Integer]
attr_reader :http_open_timeout

# @return [Integer]
attr_reader :http_read_timeout

# @return [IO, nil]
attr_reader :http_debug_output

# Fetches a given metadata category using a String path, and returns the
# result as a String. A path starts with the API version (usually
# "/latest/"). See the instance data categories for possible paths.
# `"/latest/"`). See the instance data categories for possible paths.
#
# @example Fetching the instance ID
#
# ec2_metadata = Aws::EC2Metadata.new
# ec2_metadata.get('/latest/meta-data/instance-id')
# => "i-023a25f10a73a0f79"
# # => "i-023a25f10a73a0f79"
#
# @note This implementation always returns a String and will not parse any
# responses. Parsable responses may include JSON objects or directory
# listings, which are strings separated by line feeds (ASCII 10).
#
# @example Fetching and parsing JSON meta-data
#
# require 'json'
# data = ec2_metadata.get('/latest/dynamic/instance-identity/document')
# JSON.parse(data)
# => {"accountId"=>"012345678912", ... }
# # => {"accountId"=>"012345678912", ... }
#
# @example Fetching and parsing directory listings
#
# listing = ec2_metadata.get('/latest/meta-data')
# listing.split(10.chr)
# => ["ami-id", "ami-launch-index", ...]
# # => ["ami-id", "ami-launch-index", ...]
#
# @note Unlike other services, IMDS does not have a service API model. This
# means that we cannot confidently generate code with methods and
# response structures. This implementation ensures that new IMDS features
# are always supported by being deployed to the instance and does not
# require code changes.
#
# @see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-categories.html
# @see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-identity-documents.html
# @see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html#instancedata-data-categories Instance metadata categories
# @see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-identity-documents.html Instance identity documents for Amazon EC2 instances
# @param [String] path The full path to the metadata.
def get(path)
retry_errors(max_retries: @retries) do
retry_errors do
@mutex.synchronize do
fetch_token unless @token && [email protected]?
end

open_connection do |conn|
http_get(conn, path, @token.value)
open_connection do |c|
http_get(c, path, @token.value)
end
end
end

private

def resolve_endpoint(endpoint, endpoint_mode)
return endpoint if endpoint

case endpoint_mode.downcase
when 'ipv4' then 'http://169.254.169.254'
when 'ipv6' then 'http://[fd00:ec2::254]'
else
raise ArgumentError,
':endpoint_mode is not valid, expected IPv4 or IPv6, '\
"got: #{endpoint_mode}"
def resolve_backoff(backoff)
case backoff
when Proc then backoff
when Numeric then ->(_) { Kernel.sleep(backoff) }
else ->(num_failures) { Kernel.sleep(1.2**num_failures) }
end
end

Expand All @@ -142,6 +156,7 @@ def fetch_token
end
end

# GET request fetch profile and credentials
def http_get(connection, path, token)
headers = {
'User-Agent' => "aws-sdk-ruby3/#{CORE_GEM_VERSION}",
Expand All @@ -151,15 +166,13 @@ def http_get(connection, path, token)
response = connection.request(request)

case response.code.to_i
when 200
response.body
when 401
raise TokenExpiredError
when 404
raise MetadataNotFoundError
when 200 then response.body
when 401 then raise TokenExpiredError
when 404 then raise MetadataNotFoundError
end
end

# PUT request fetch token with ttl
def http_put(connection, ttl)
headers = {
'User-Agent' => "aws-sdk-ruby3/#{CORE_GEM_VERSION}",
Expand All @@ -174,10 +187,8 @@ def http_put(connection, ttl)
response.body,
response.header['x-aws-ec2-metadata-token-ttl-seconds'].to_i
]
when 400
raise TokenRetrievalError
when 403
raise RequestForbiddenError
when 400 then raise TokenRetrievalError
when 403 then raise RequestForbiddenError
end
end

Expand All @@ -191,9 +202,22 @@ def open_connection
yield(http).tap { http.finish }
end

def retry_errors(options = {}, &_block)
max_retries = options[:max_retries]
retries = 0
def resolve_endpoint(options)
return options[:endpoint] if options[:endpoint]

endpoint_mode = options[:endpoint_mode] || 'IPv4'
case endpoint_mode.downcase
when 'ipv4' then 'http://169.254.169.254'
when 'ipv6' then 'http://[fd00:ec2::254]'
else
raise ArgumentError,
'`:endpoint_mode` is not valid, expected IPv4 or IPv6, '\
"got: #{endpoint_mode}"
end
end

def retry_errors(&_block)
attempts = 0
begin
yield
# These errors should not be retried.
Expand All @@ -202,22 +226,14 @@ def retry_errors(options = {}, &_block)
# StandardError is not ideal but it covers Net::HTTP errors.
# https://gist.github.com/tenderlove/245188
rescue StandardError, TokenExpiredError
raise unless retries < max_retries
raise unless attempts < @retries

@backoff.call(retries)
retries += 1
@backoff.call(attempts)
attempts += 1
retry
end
end

def backoff(backoff)
case backoff
when Proc then backoff
when Numeric then ->(_) { Kernel.sleep(backoff) }
else ->(num_failures) { Kernel.sleep(1.2**num_failures) }
end
end

# @api private
class Token
def initialize(options = {})
Expand Down
Loading
Loading