[jetty-users] Jetty Client testing with Iūdex crawler

[David Kellum <dek94@xxxxxxxxxxxxx>]

Hi,

I've been working on a Jetty Client integration with the Iūdex web crawler (to replace use of the venerable apache httpclient 3.1)  I'm finally at a point of starting end-to-end testing: the performance numbers and reduction in necessary threads is highly encouraging, but I am having some troubles around redirects and some other stability issues at volume/long runs. I've filed some bugs on jira.codehaus.org.  Hope that is ok. I waited for most the day for eclipse bugzilla to send me a login but it hasn't. 

I believe I can provide some valid fixes for the RedirectListener, to resolve these bugs:

http://jira.codehaus.org/browse/JETTY-1420
Exchange onException,onConnectionFailed not called on errors with RedirectListener

http://jira.codehaus.org/browse/JETTY-1421
RedirectListener sends wrong Host header for cross-host redirect

Would a topic branch on fork of github.com/eclipse/jetty.project be appreciated to this end?

Then for the following I'm completely stumped and need some help:

https://jira.codehaus.org/browse/JETTY-1422
IllegalStateException: 0 => 8 in Client

https://jira.codehaus.org/browse/JETTY-1423
Consistent RejectedExecutionException after running for some time

At a high level, any suggestions for what I can do to help get these resolved? 

Finally, in case anyone is interested in what I'm doing in my own code base:

https://github.com/dekellum/iudex/tree/async-executor-service

And possibly of particular interest:

https://github.com/dekellum/iudex/blob/async-executor-service/iudex-jetty-httpclient/src/main/java/iudex/jettyhttpclient/Client.java
https://github.com/dekellum/iudex/blob/async-executor-service/iudex-jetty-httpclient/lib/iudex-jetty-httpclient.rb
https://github.com/dekellum/iudex/blob/async-executor-service/iudex-jetty-httpclient/test/test_httpclient.rb
https://github.com/dekellum/iudex/blob/async-executor-service/iudex-http-test/lib/iudex-http-test/test_app.rb#L75

Thanks for all the working on Jetty both service and client.  I'd really like to help see the Jetty client work out for my project.

--David