Problem pulling through http proxy

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem pulling through http proxy

toni.arte
Hi all,
 
I’m doing some performance testing on Mercurial, using the Linux kernel Mercurial repository as the test data.
 
I noticed a problem with the http proxy setup:
* I have no problems cloning the whole repository over a proxy connection
* But if I clone the repo up to a certain revision, any attempts to do a ‘hg incoming’ or ‘hg pull’ fails
* This happens on any version of Mercurial I have tried (including 1.5.3), and it’s the same on Windows and Linux
* Connecting directly into the Internet solves the problem, but that is not possible within our office premises
 
Here is an example:
 
$ hg clone --noupdate http://www.kernel.org/hg/linux-2.6 -r v2.6.34
requesting all changes
adding changesets
adding manifests
adding file changes
added 192213 changesets with 921413 changes to 49397 files
 
searching for changes
abort: HTTP Error 400: Bad Request
 
35 total queries
sending capabilities command
capabilities: unbundle=HG10GZ,HG10BZ,HG10UN branchmap lookup stream=1 changegroupsubset
sending changegroupsubset command
abort: HTTP Error 400: Bad Request
 
searching for changes
abort: HTTP Error 400: Bad Request
 
35 total queries
sending capabilities command
capabilities: unbundle=HG10GZ,HG10BZ,HG10UN branchmap lookup stream=1 changegroupsubset
sending changegroupsubset command
abort: HTTP Error 400: Bad Request
--
Toni Arte
 

_______________________________________________
Mercurial mailing list
[hidden email]
http://selenic.com/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Problem pulling through http proxy

Mads Kiilerich
[hidden email] wrote, On 05/24/2010 02:42 PM:

> Hi all,
> I’m doing some performance testing on Mercurial, using the Linux
> kernel Mercurial repository as the test data.
> I noticed a problem with the http proxy setup:
> * I have no problems cloning the whole repository over a proxy connection
> * But if I clone the repo up to a certain revision, any attempts to do
> a ‘hg incoming’ or ‘hg pull’ fails
> * This happens on any version of Mercurial I have tried (including
> 1.5.3), and it’s the same on Windows and Linux
> * Connecting directly into the Internet solves the problem, but that
> is not possible within our office premises

It seems like the proxy chokes on the Mercurial http traffic. Is it
filtering aggresively? Could you check the proxy logs?

It could be related to big http requests - see
http://mercurial.selenic.com/bts/issue1650 (and perhaps
http://mercurial.selenic.com/bts/issue682 and
http://mercurial.selenic.com/bts/issue2126)

/Mads



_______________________________________________
Mercurial mailing list
[hidden email]
http://selenic.com/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

RE: Problem pulling through http proxy

toni.arte
>[hidden email] wrote, On 05/24/2010 02:42 PM:
>> Hi all,
>> I'm doing some performance testing on Mercurial, using the Linux
>> kernel Mercurial repository as the test data.
>> I noticed a problem with the http proxy setup:
>> * I have no problems cloning the whole repository over a proxy
>connection
>> * But if I clone the repo up to a certain revision, any attempts to do
>> a 'hg incoming' or 'hg pull' fails
>> * This happens on any version of Mercurial I have tried (including
>> 1.5.3), and it's the same on Windows and Linux
>> * Connecting directly into the Internet solves the problem, but that
>> is not possible within our office premises
>
>It seems like the proxy chokes on the Mercurial http traffic. Is it
>filtering aggresively? Could you check the proxy logs?
>
>It could be related to big http requests - see
>http://mercurial.selenic.com/bts/issue1650 (and perhaps
>http://mercurial.selenic.com/bts/issue682 and
>http://mercurial.selenic.com/bts/issue2126)

Straight to the point Mads...

Mercurial will send out a LONG request (~7 kbytes), and the proxy will answer with '400 Bad Request'. I have no idea on which web proxy we are using, but all the three ones our automatic proxy configuration script gives behave the same.

The request in question is:
---
GET http://www.kernel.org/hg/linux-2.6?bases=f8f1e8206aff6c146399321798bb695747ff66ac+8bf93ff0be2b8a21afd27fb14bb8a78f46fd1c7d+11b77853803f04b62382f5e9a410a22802e094ca+6155560dbdf2972b3b5d0c268ce0c893b536007a+158cddb356725636763784c8961ee70412c6aa08+ec2bb617cee07e548e9c20aa71a2a48bffa4f131+a76b6e0861bf2bbc27b241530f258d07bde15c04+51c1216abdea8c3920eff21c4b3cd5ec8d59fb31+78d9f9f1f30cc0794b1b4b25ed1ebf6301b340f6+910cef9677413e6a2be37c73efe176d272d2bca8+5250cd1c3160dc388a2f279a7d0adba58d37f3df+4f00bc51e5a6739f48638a7c972225b6803d20ec+6c2ef32c8e41a4be98861052f256206f7cdf9141+dbd660338ef118671709b2bd1af21e8924cce380+b70edd3e09e15df5d78ebbc3f38e70b66da47548+7ee44d8622e6774b6839df84453615755abca1b8+477e72d2fbfecae0592669475570799306dcd92a+de3be1ed41dbfdbf9248a2603684acbc1a50370a+b50bb03282010af19a2f301858ee1c52394a447f+943f018379fb8d54942da3627f21f58b113fd190+bdff8cd98b1804d42a2f6558850845dfe16f6e8d+3db822bc2d9e88d0e0d8b271108c768d9bcaa0d3+c8195511c15912c0c88c2c393dfc7d94fbcc972a+0a
 1c236c33d1b28f309fb776d6c1268ffa5eec09+0a251c8a0d1dfd073016f4fac69518ec939ac2f9+c18a66120db93eb9eafe1ba041d78afa1ab0d0d5+0ae74fad9ca91dd2127138920d5bfd87a607fceb+555f192b5ad07c7c8dd61017c5428ac8076aa249+363f041aeb2a3b385c83ce4de2b6556f5e20c230+9fafb61653db9e0a97ccea1ee302583bad4a2769+fc04755542ebf587617815f9e7a555d62f803dab+5a02c59c9c9a93125303f1a3effaeecabbce7664+bb70bc69a552011716285a209b1a47ea03093d8d+64c03ec56b4a14ac81dcc0b53d45fafc50bb2d24+f9e2c8610bde9d9b00caa8341c14a0f930f5e0de+ec5abd9f425992ee79b2e7ffa0e5105471d3596d+d9faf43f7f816b9e10a34517da9def47dc1dc62f+ff2b7200a7c95558727003cfbe69d767130b5dcf+35974b321baf4cce5edff6a813895ecb6060eebf+8efa168e3244bd5a4bc0ef1e6aa65ab43832f25e+30a9f4b5e094e78ded8a2881e01cee2b9b4909a2+ce7f67f937bcec497052b5f60eefd55e7df1cc2d+0f4fdc23e13c75174fe1cc272cef18e4c44a9023+1dabd3c234ade2b9adf0806b5ebb05d3dd640503+73d4827657f84af477dac1b7c7d31ea6310cd8d6+33a8f256c9c443a74929ec98467e1e4645d61142+ea7c77369b04293b681c5e4562135393061fecf9+eff33e7
 3bb95672af5682720110223f76a36b10b+9b8037ebbb0786c13b7e8df6d3eb41661d6164d2+f7f43a2ea4673839e40a0c35517adcfb92fa5910+474ac0e96a87b72cdc170480b32ba0593e828d18+54a13710eb88e80cbb5aea5bf1ddda6ef4c715f2+5099f93d986f50e0148db080ab1a5b61767e1fbe+725d3e4e68aa62596d1da4b4777f4c8e84040591+bdfaaace424e84edd17641ba1593210c73323d2c+5f20c85d60833cd44c789415b29ecc0720d6775f+7ee177bf3b4db65535d4a053c790d5382155d600+10026a6f544e9ca87ca8f994208e97f989c6c666+2f52bb011b3e841c78188273af4f3721dc741d9f+de77eb04dee764671bedd01c57721a62e5daedbe+5246aaeb5f5ab692953f1076b23a8ebf6c51c3a6+f07588c2169ce7aa2cd2b19cdf561ad34fcad97d+ea7032d815b2f170d6ba1ff30bb58bedf33ecb97+01ac8ceea9012229ebc99da74f3a06a21d7fa74a+59ccf721c92b3cf4d5d2dbd8125a1ac9e65e567c+70fc925d56a7af3ad0d1ea1b417e1d55ea9ee240+fdb2402f64e46c0991e2fb78e8bf933de540db74+cb7e35b9995d486dbc0904b85753dd6896df7a3c+951d9bf078800308c33bf1620730e15c127d3a56+24a4d72af13c71702e11bec456bcbd0e57ccf542+6d86a2b2bd3460f9a0c9d2145ed4732dfd733916+80c1e0bfcd4d
 8cae91acba8dceb730daf558d961+58b2350a02addd04cc69354dd617bdd03710d5fb+f7d1da47f2d95b7f39ed21ec4d510a7515e050b3+37ee92e6ca020bb386485641760754d6b9086431+dfe6b477c5df9a8be8fecce84c1a0147ad31e6d2+59e226fb3d32542e2a30b905976cd0aa97af443e+a78e9ce758eeba7a1b3b385106b7df99975ec015+324e15a715ad559735a478bc752f3d9524b0a3b5+f48b6096cdf298389dfd9f56fedf717e51b7a6f4+f704e6e02e73d552c7f99273c7c4ad5543fcc61e+babe495c300ff41b50ad77c6bb522c4190ff5b22+450a1c7204767fd1a86568ce651b26e10c9fad00+23dee2669a1da57986d10f6dcf4f5846fae53714+ddf77d8935a18363a4a3de5337bb33e9dd094a82+1bf6078d66838e53cea4b4a5c15c35b1e5d71047+3aab205658d9a86ba80e2e52abce1b8446c1fad3+6e6e41645be2852019cdd513904f24fd7ac38a14+4d6b3699c5d5d95060d1d7cdb30cfea117460fe6+8e0753bbc116d4d6e9300bfbc08b995920cdcc5b+cc911f787d96c12757b52146b53005dfee8d89c2+7138aa9bdff11f622b2d159563340633ee4964d5+ea91644eb6d2ce398c92827d5ccb962cd859f090+7e5addf7050c1b5c7da74778d942b395952b94d9+c8b2985e4d6b2ee672ff4dfb11fdc57282f03fb0+ba067d9567cb5cdd9
 3f0264cefbb4c50b4b64cdd+27731114ba6e3aa6565f99d8ca86140f579989fb+79b0ad4865a8972299d3e8c5b9a65fd11d467059+be85502a7514c46f7bd8cded763f36444357edf3+9dac7756a9214021f55b0bbb54392cd4877de787+7e033ca1fc3f4c58cca2679aa9b049e667b1a0bf+0a1373d03c7760ce4b75a51fef4493c0d8e69611+c4b52471734bc3d0556f2f92a060a85670c79cf4+b2fb15b57b8191c20cc987dce502b5c3566890ab+cacfe776965afe69ee6bf71c4d4e65c2b96e7ae3+36c46614c9d2a720bd267bc5f226978256d2d38c+9c1e9c13915509870902c4e2d3c4d3b47911a59f+c1d59a8084445df153a3725e05b3dd19428312ed+a2c37434b731671ac20fdf3d17ad00978e3a759e+d567f44766684184c0b8b4a7469dba53a35b093e+ca4099710bdc4a886970ba416e1f870a0d54a3be+246698acef6e99d595d2ba5c7e7d638b1072b748+e8f58eb6ce55cb44da5844778ba58de7402a2d5d+071fabbd85610eac23cdd46bbb25a75d75fa32c5+82f53569cdd3fceec91a9825deb3ffdcb797604d+fcd12255451732533114170e05ee7e7f6602be80+141305b5574a4c592ee55f5a0cf547401f4dd38a+c01cb90e42a94e8860962efc2ecc2f7bbb1fb843+db210569a9e6bb3da6d97c5a20e76fa3748f4bd7+d02cc959f6b419153cd707
 cb00510971e6258ca4+57325c6a00509dc1ae9e7b9937a90b070bd50a7f+113fe04ccd07c70972c75f16164c7becc36ca41f+c20acfe408bfb87ec079cb17a74989448ebfb4cc+b489d8a20d4011c7d4925cbcdd089abedb6beb70+ef05aa6e831cb3b0d2f368f43942b8acab7f0259+72f043492974ef7479bd3cfa0451c334eea49c95+7f6a3d9e72d8da6c7d448c06747460992bf98f71+5b4e03577d9b95078b07f821472b0592d7355414+a70a5b5202ae4f78acf28c3b739b7688ea2acc6f+06c6cd29032c64c464bad0c114e2e0ca8fd0840f+009fe08e06d3e53fdbf7ce1eb077793f8ff49239+d69cf5e3c3642895e607640f4522023dc9da8565+248cab40ff7623de87b4f544c8b4b317a2952248+b42701219f2af60b029291687b68982e3afae320+4f688775db1e9755fb5e48f1d0e74849d460be49+aea390bab48ed37a9be1efedde2701d934da88db+9d81ac05944b6afb28db7ea529c067b5a222a0ec+7161f3a06168c00e2a1ce1c3fa159d7f524b0e4b+98b454cf0e46659561bb0797f7649c92ea951f8e+164651e48ee9d0760b61651b2055784b18dfbab5+2d4e8c7bf3d31febe24ec7fb580a018bbe95ad1d+6e233607073393549e25bfbcd946647df4065ab3+5b60110c980fe2ece8e2512df16566e2d396b0b1+50c4780a4bb12578c286609496f
 8fe78df138d48+31512a6c415d46fae03916f7119a9e02ff4e15c9+83a41bf48b2aab6220c576f3d6a9b7044858bec2+9aa91b7aa49201a179d73ebea391bbc5c47178f4+c2c9617e83d53e8bc02a1e9c818a66a6a91eddee+78d22ba68ee4437ab87af9e2ad640e32303a39da+e83ac968cca7fbe1ab26aa32dddd061c7af1bc2a+0a8b112076af5b00bd395a32e8b42bce354f01f6+e3e6736c3fb0ef4c6cddd9e52651b14614aee8ca+897aad8cf0234d731cb5a9f0fa4d486f88e6db5c+2afff7e9b4733274358d4e4f6e67f21220f0efd4+9bbad10a0ec40219242019c3fc5d4639e1f40ab1+44c13ab1f1e69006d058329f17da8f713fa81de5+238a01e19af48e4605db35f085bbd4b28bd3004f+eade0f355c5638f66f0fd48e27cd8fd5191572a7+41c215caf79af38dacc461e70303600fe7863510+9610ccd4f8f1a7f7ece366374d41c24c9019953d+476d2275b63973cfa138983897dbb04ab1ccae4d+854f4d46781868fde46ee78924a859d8617d37dd+094e515621102e7a9aa6cf93bfe930dad4a6f8d4+296d64364e618be4a3d014a51ccb0930d725a021+8a2f46080755c630269af507b940949b8464c499+8d6ee68e987d84be8843db3136c1edbd2e7ef5b1+d909e2e915b0167860c158026db0f511841367af+7b2f29ceaa21e089b9dc9f469a784a54
 d79b9eb3+0ab5a99c7fdf3bbbe7ff6b21c2f4c5ae429cc824+14edd772f5e981bf6f66a6b1e9e3b9325e796ead+0b4dd66f58c24648b85890a1764720d493955d29&cmd=changegroupsubset&heads=7718a91b01dbaa40144e6f2b51b1b94bc43898c4 HTTP/1.1
Accept-Encoding: identity
host: www.kernel.org\r\naccept:application/mercurial-0.1
user-agent: mercurial/proto-1.0
---

And this is the reply from the proxy
---
HTTP/1.1 400 Bad Request
Cache-Control: no-cache
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Proxy-Connection: close
Connection: close
Content-Length: 691
---


--
Toni
_______________________________________________
Mercurial mailing list
[hidden email]
http://selenic.com/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

RE: Problem pulling through http proxy

toni.arte
>From: [hidden email] [mailto:mercurial-
>[hidden email]] On Behalf Of Arte Toni (Nokia-D/Salo)
>Mercurial will send out a LONG request (~7 kbytes), and the proxy will
>answer with '400 Bad Request'. I have no idea on which web proxy we are
>using, but all the three ones our automatic proxy configuration script
>gives behave the same.

The email gateway broke the HTTP request into multiple lines. It really
is a single line request, ~7 kilobytes in length.

I did some further study on this, and it really seems to be a configuration
issue on the http proxy server. Most web servers seem to have a
configuration variable for the maximum request length.

In the case of Apache, this actually isn't that far off from the default
Maximum request length on Apache (the default value is 8190 bytes):

http://httpd.apache.org/docs/2.2/mod/core.html#limitrequestline

So, you may very well run into a case where you cannot run
'hg incoming' or 'hg pull' from a large repository at all. This of
course only happens on a case when you are missing a large number
of changesets, on the default Apache configuration.
--
Toni



_______________________________________________
Mercurial mailing list
[hidden email]
http://selenic.com/mailman/listinfo/mercurial