Repo clones over HTTPS are broken for slow connections

[ell1e]

Select Topic Area

Question

Body

Introduction: Repo clones over HTTPS are broken for slow connections due to what seems to be an error on GitHub's end, and GitHub apparently doesn't plan to fix it. This is can be a big issue since e.g. this also breaks any submodules or install scripts that indirectly access any GitHub repo of any kind that isn't trivially sized.

I am writing this here so people can reference this issue.

This is who is affected:

• Your connection is reliable enough to download 100MB+ files without a hiccup or need to pause and resume in at least the majority of cases.

• You can clone big repositories on gitlab.com and codeberg.org and other alternate git repository platforms without issues, other than at most rare hiccups which don't repeat on 2nd try.

• But your connection is DSL era slow, 5mbit/s or slower.

This is how it looks like:

$ git clone https://github.com/FortAwesome/Font-Awesome
Cloning into 'Font-Awesome'...
remote: Enumerating objects: 87765, done.
remote: Counting objects: 100% (12255/12255), done.
remote: Compressing objects: 100% (5196/5196), done.
error: RPC failed; curl 92 HTTP/2 stream 5 was not closed cleanly: CANCEL (err 8)
error: 4872 bytes of body are still expected
fetch-pack: unexpected disconnect while reading sideband packet
fatal: early EOF
fatal: fetch-pack: invalid index-pack output

This is why it seems to be an error on Github's end:

• I found others with the same problem, although due to people usually having faster internet it's somewhat rare.

• Cloning similarly sized or magnitudes larger repositories with HTTPS from any other platform like gitlab.com works fine, just github.com is affected.

• Cloning the repositories from GitHub via SSH instead works.

• Downloading anything else, like archives, of even larger size via HTTPs from GitHub works. Only HTTPS git clone doesn't.

• GitHub's support admitted in response to me that this is likely a problem on their end, but that they don't plan to fix it any time soon. At least this is my understanding of their responses.

[ell1e]

This is still broken as of today, 2024-11-07.

[riv19]

Yes, not possible to clone repositories on 4G/5G networks. Are these networks outdated ?
And don't try to configure the client side with "http.postBuffer" and similar - doesn't help.

[ell1e]

I think it's somehow related to the server having a timeout if in time window X there were less than Y bytes transferred. And this timeout seems to have a really high threshold to clear. This means if the connection isn't like, 20mbit/s or faster, the server will just kill your connection even though it's making progress steadily. At least that's my best guess what could be going on.

[MarinoJurisic]

What in the world is this? Some horrible DoS mitigation?

GitHub's support admitted in response to me that this is likely a problem on their end, but that they don't plan to fix it any time soon

Could I see their response?

This issue can be reliably reproduced by attempting to clone a repository from github.com over HTTPS over Tor. However I have also reproduced this issue by attempting to clone a large repository through connecting to an external SOCKS5 proxy over Tor where the target system hosting the proxy is on a fast network and cloning on the target system itself succeeds which would led credence to your theory.

[MarinoJurisic]

That or something related to load-balancing would be my guess. But ultimately I agree with @ell1e here.

[MarinoJurisic]

Sounds like this other discussion is a duplicate of this issue.

[MarinoJurisic]

Apparently having a ticket with GitHub Support remain inactive for too long (>3 weeks) sets it to 'archived' which renders it closed. Hopefully just a small bump in the road.

Created a new ticket with reference to my previous ticket.

[ell1e]

The amount of bureaucracy they impose on you for what seems to be something like a load balancer bug is starting to become absurd. 🫥

[MarinoJurisic]

I was somewhat optimistic about the last response I received but hopefully this will not set me back to square one.

To make matters worse I was not informed of the impending status change of my previous ticket.

[nikoraasu]

Same issue here, it's really frustrating

[ell1e]

If you contact GitHub support to report a git clone issue with reference to ticket number 2835344 there's a chance they'll eventually be bothered enough to do something about it. From my limited understanding, they seem to be aware of the issue but under the impression that it's not worth fixing for now.

[danigar]

We are experiencing the same issue here since the end of the last week. In our case almost every clone or fetch done through HTTPs is extremely slow (around 1 Mb/s). It's causing all our CI pipelines to fail due timeouts.

In some cases we are seeing the same output error:

fetch-pack: unexpected disconnect while reading sideband packet
fatal: early EOF
fatal: fetch-pack: invalid index-pack output

[MarinoJurisic]

All I can suggest is to

1. create a throwaway github.com account and use SSH for CI repository clones (it seems to be unaffected by this issue) or

2. Contact github.com support and press them on the facts in this discussion and reference the ticket numbers to (hopefully) escalate the priority of this issue and its resolution.

[mhaddon]

We are also having this issue in our CI pipelines hosted on Google Cloud. We are also having extremely slow transfer with SSH. The node itself does not have general networking issues, and the issues are specific to Github.

[MarinoJurisic]

In consideration with the previous discussion and your experience it may be prudent to file another ticket with GitHub support making note of ticket numbers and this thread.

[keyvez]

Very slow for me too, and continues after weeks, git clone https://github.com/ltdrdata/ComfyUI-Manager comfyui-manager Original discussion
: https://github.com/orgs/community/discussions/149702#discussioncomment-12020530 and here is a demo of the slow speed, this happens 8/10 times.

Here are my speed results, so I am not on a slow connection by any means, I have a fiber connection, and ethernet connection to this server, and everything else is blazing fast, even github zip download is fast:

[MarinoJurisic]

The relevant factor here appears to be the throughput of a Git operation over HTTPS transport. Not the speed of your underlying connection.

[ell1e]

This seems to have been fixed at some point during the last days leading up to today (2025-03-21), thanks in part to @MarinoJurisic 's tireless efforts to convince Github support to revisit this problem!!! 🎉

[etbuira]

I still have the issue