Apt-mirror on packages.graylog2.org fails

Graylog Central (peer support)
1

Problem:
I’m having some problems letting apt-mirror make a copy of the Debian repo, so that we can use it within our datacenter.

I noticed that the URLs at ://packages.graylog2.org/repo/debian/… all redirect to ://graylog2-package-repository.s3.amazonaws.com/debian/…

apt-mirror’s behaviour is to use wget’s --timestamping option to check if the remote file is newer. This works, except that the S3 URL expires in the 1 second wget needs to first determine that the file is newer, and then starts downloading the file.

Potential solution:
Could packages.graylog2.org consider increasing the S3 URL expiry, even if it’s just a few seconds? Letting apt-mirror do its job would help us/you decrease the amount of S3 bandwidth consumed.

Here is the output from apt-mirror/wget:

--2017-09-07 04:10:04--  https://packages.graylog2.org/repo/debian/dists/stable/2.2/binary-amd64/Packages.gz
Connecting to web-proxy.abc (web-proxy.abc)|1.2.3.4|:80... connected.
Proxy request sent, awaiting response... 302 Found
Location: https://graylog2-package-repository.s3.amazonaws.com/debian/dists/stable/2.2/binary-amd64/Packages.gz?AWSAccessKeyId=AKIAIJSI6MCSPXFVDPIA&Expires=1504729205&Signature=L7fBApigxWBz6d0FIuDffd6AVpY%3D [following]
--2017-09-07 04:10:05--  https://graylog2-package-repository.s3.amazonaws.com/debian/dists/stable/2.2/binary-amd64/Packages.gz?AWSAccessKeyId=AKIAIJSI6MCSPXFVDPIA&Expires=1504729205&Signature=L7fBApigxWBz6d0FIuDffd6AVpY%3D
Connecting to web-proxy.abc (web-proxy.abc)|1.2.3.4|:80... connected.
Proxy request sent, awaiting response... 200 OK
Length: 3665 (3.6K) [application/gzip]
Remote file is newer, retrieving.

--2017-09-07 04:10:07--  https://graylog2-package-repository.s3.amazonaws.com/debian/dists/stable/2.2/binary-amd64/Packages.gz?AWSAccessKeyId=AKIAIJSI6MCSPXFVDPIA&Expires=1504729205&Signature=L7fBApigxWBz6d0FIuDffd6AVpY%3D
Connecting to web-proxy.abc (web-proxy.abc)|1.2.3.4|:80... connected.
Proxy request sent, awaiting response... 403 Forbidden
2017-09-07 04:10:08 ERROR 403: Forbidden.
2

Thanks for bringing this up!

I’ve created a feature request in our internal GitHub repository for the package repository server.

3

The problem isn’t expiration of the token (which is valid for 10 minutes), but that the client is using different HTTP methods (HEAD, then GET).

S3 allows signing requests but requires exactly one HTTP method to be signed for:
https://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html

In other words, an access token for a HEAD request is not valid for a GET request.

4

Ah! You’re right, I must’ve either been looking at an older Expires parameter or missed the minutes entirely when I was checking the timestamps >_<

Is there a possible fix for this? It seems like the package repository server has to respond to the HEAD request with appropriate Last-Modified header instead of redirecting to a pre-signed URL, and only redirect for the actual GET request.

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.