Skip to content

S3 Link Issue: Document indexed but access fails #4486

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
MagnusHL opened this issue Apr 9, 2025 · 3 comments
Open

S3 Link Issue: Document indexed but access fails #4486

MagnusHL opened this issue Apr 9, 2025 · 3 comments

Comments

@MagnusHL
Copy link

MagnusHL commented Apr 9, 2025

I’m using the S3 connector with Onyx to index documents from an Amazon S3 bucket. Indexing works fine, but when I try to access a linked document, I get an error. The document itself is accessible if I open it directly from the bucket (public access is enabled), so this doesn’t seem to be a permission issue.

Is there something wrong with how the S3 link is being generated? Should the connector explicitly address the bucket when generating the document URL?

Here’s the error message I get:

PermanentRedirect
The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
debitorlieferscheine.s3.eu-north-1.amazonaws.com
debitorlieferscheine
955X3QG2BQC2B0YR
8ytj/rETMUCPehKXmWIDa2ffHLYTmA2/fMOo5NldKUclbg8NuCFMnlnsvHzDd9DEWxPP56EvR3E=

@MagnusHL
Copy link
Author

Update / Root cause identified

S3 now enforces region‑specific endpoints. The bucket debitorlieferscheine lives in eu‑north‑1, but the connector builds URLs that point to the default s3.amazonaws.com (us‑east‑1). Hitting such a URL returns a PermanentRedirect to debitorlieferscheine.s3.eu-north-1.amazonaws.com.

Broken URL returned by the connector
https://debitorlieferscheine.s3.us-east-1.amazonaws.com/_0G0G6_S25C-0i25041108440_0008_OCR.pdf

Working URL (same object, correct region)
https://debitorlieferscheine.s3.eu-north-1.amazonaws.com/_0G0G6_S25C-0i25041108440_0008_OCR.pdf

Expected behavior
The connector should generate region‑correct URLs, or pre‑signed URLs that include the bucket’s region.

Workarounds
• Configure the connector’s AWS SDK with region=eu-north-1
• Or front the bucket with CloudFront / a redirect bucket in us‑east‑1`

@dhubai
Copy link

dhubai commented May 4, 2025

Thanks for the explanation.

Will this enable me to view the documents directly within the Onyx UI, or will it still just open the document link in a new browser tab?

@MagnusHL
Copy link
Author

MagnusHL commented May 5, 2025

It is just a URL and you can view the pdf inside your browser, so direct access

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants