Bug 506935 - Custom user agent for javadoc http connections
Summary: Custom user agent for javadoc http connections
Status: NEW
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 4.6   Edit
Hardware: All All
: P3 minor (vote)
Target Milestone: ---   Edit
Assignee: Jay Arthanareeswaran CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords:
Depends on:
Blocks:
 
Reported: 2016-11-02 15:10 EDT by Andras Kerekes CLA
Modified: 2022-07-27 15:19 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andras Kerekes CLA 2016-11-02 15:10:17 EDT
org.eclipse.jdt.internal.core.JavaElement.getURLContents(URL, String) does not set a user agent, thus uses the default one set by the JDK. Some CDN providers (CloudFlare is one of them) by default uses a browser integrity check (https://support.cloudflare.com/hc/en-us/articles/200170086-What-does-the-Browser-Integrity-Check-do-) that filters out requests that have 'Java/*' as user agent (and possibly others as well) and responds with an HTTP 403. It would be useful, if the request would set a different user agent, e.g.: 'Eclipse JDT/<version>'.

Possible workarounds:
1. use -Dhttp.agent="<something other than Java/*>"
2. get in touch with the site's maintainer that is hosted on the CDN and ask them to turn off/customize this check
Comment 1 Jay Arthanareeswaran CLA 2016-11-03 01:34:25 EDT
(In reply to Andras Kerekes from comment #0)
> It
> would be useful, if the request would set a different user agent, e.g.:
> 'Eclipse JDT/<version>'.

This sounds reasonable to me, although it will just be a dummy user agent, as what eventually renders the HTML is the system's browser.
Comment 2 Brian de Alwis CLA 2016-11-03 11:43:45 EDT
This can currently be seen when setting a Javadoc location for a library to something hosted on static.javadocs.io.

$ curl —silent --show-error --head -A Java/1.8 \
   'http://static.javadoc.io/com.googlecode.objectify/objectify/5.1.13/com/googlecode/objectify/ObjectifyFactory.html'
HTTP/1.1 403 Forbidden
Date: Thu, 03 Nov 2016 15:41:16 GMT
[snip]
Comment 3 Manoj N Palat CLA 2018-05-21 06:13:46 EDT
Bulk move out of 4.8
Comment 4 Eclipse Genie CLA 2020-08-01 02:36:13 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.
Comment 5 Eclipse Genie CLA 2022-07-27 15:19:46 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.