Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
RE: [smila-dev] RE: JDBC crawler 
Hi Juergen, 

the JdbcOdbc-Bridge seems to choke on leading whitespaces and/or linebreaks in the SQL statement: 

	<SQL>SELECT ID, Description FROM UsedCarsDescriptions</SQL>

works fine whereas 

	<SQL>
		SELECT ID, Description FROM UsedCarsDescriptions
	</SQL>

produces an empty resultset (which results in zero records being created, of course). The Derby driver does not seem to have this problem: it runs fine with both versions.

The SQL is stripped of leading whitespaces now and I got about 10k records from your database. I added some Testcases for JdbcOdbcCrawling to the Test-Bundle as well, but the are not active (outcommented) as they would probably break the build with the data source not being present on Bamboo. They should work on your machine though.

Kind regards, 
Michael



-----Original Message-----
From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-bounces@xxxxxxxxxxx] On Behalf Of Juergen.Schumacher@xxxxxxxxxxx
Sent: Donnerstag, 27. November 2008 10:55
To: smila-dev@xxxxxxxxxxx
Subject: RE: [smila-dev] RE: JDBC crawler

Hi Michael,

here you are: two logs with TRACE for the JDBC crawler.
one with grouping enabled and one without grouping.
There is a nice exception in the grouping run, hope it helps. I can send you the database file off-list if you need it.
The run without goruping doesn't seem to produce an error, but it also does not produce records, it seems (-;

Cheers,
Jürgen.


> -----Original Message-----
> From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev- 
> bounces@xxxxxxxxxxx] On Behalf Of Michael Breidenband
> Sent: Wednesday, November 26, 2008 3:09 PM
> To: Smila project developer mailing list
> Subject: [smila-dev] RE: JDBC crawler
> 
> Hi Juergen,
> 
> problem no.1 should be fixed - I just commited the fix to svn.
> 
> I could not yet reproduce the second problem though. my initial 
> assumption was obviously incorrect: your usage of the grouping element 
> in the index-order you posted seems to be in perfect order (work on 
> the documentation page is in progress). I tuned up the 
> logging-statements a
> bit: could you run your crawling job again with the current version 
> from the trunk and with the log-level set to TRACE (at least for
> org.eclipse.smila.connectivity.framework.crawler.jdbc) and send me the 
> log file? alternatively you could provide me with the database schema 
> you used ...
> 
> Best regards,
> Michael
> 
> 
> -----Original Message-----
> From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev- 
> bounces@xxxxxxxxxxx] On Behalf Of Michael Breidenband
> Sent: Dienstag, 25. November 2008 09:26
> To: Smila project developer mailing list
> Subject: [smila-dev] RE: JDBC crawler
> 
> hi,
> 
> the first problem is very likely a bug (Mea culpa). I shall check this 
> and provide a fix ASAP.
> 
> the second problem seems to be related to improper use of the 
> <Grouping> element - very likely caused by the veritable lack of 
> documentation about the format of the IndexOrder. I shall provide a 
> wiki page - likewise ASAP. could you post the IndexOrder you were 
> using?
> 
> 
> Kind regards,
> Michael
> 
> 
> -----Original Message-----
> From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev- 
> bounces@xxxxxxxxxxx] On Behalf Of August Georg Schmidt
> Sent: Montag, 24. November 2008 15:21
> To: Smila project developer mailing list
> Subject: [smila-dev] RE: JDBC crawler
> 
> Michael is currently ill. I'll notify him when he is back. This should 
> be an error.
> 
> -----Original Message-----
> From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev- 
> bounces@xxxxxxxxxxx] On Behalf Of Juergen.Schumacher@xxxxxxxxxxx
> Sent: Montag, 24. November 2008 15:19
> To: smila-dev@xxxxxxxxxxx
> Subject: [smila-dev] JDBC crawler
> 
> Hi,
> 
> I've tried to use the new JDBC crawler in SMILA, but I did not succeed 
> yet.
> Is there any documentation about the IndexOrder format? I did not find 
> something in the Wiki.
> After fixing the DS declarations to get the crawler service registered 
> to the CrawlerController (have committed it), I'm still running into 
> one of two problems:
> Without a <Grouping> in the <Selections> element, I get
> 
> jdbc.JdbcCrawler                              - Encountered
> RuntimeException in ProducerThread
> java.lang.NullPointerException
> 	at
> org.eclipse.smila.connectivity.framework.crawler.jdbc.JdbcCrawler.popu
> l
> ateRetrievalResultSet(JdbcCrawler.java:624)
> 	at
> org.eclipse.smila.connectivity.framework.crawler.jdbc.JdbcCrawler.hasN
> e
> xt(JdbcCrawler.java:492)
> 	at
> org.eclipse.smila.connectivity.framework.crawler.jdbc.JdbcCrawler.acce
> s
> s$7(JdbcCrawler.java:487)
> 	at
> org.eclipse.smila.connectivity.framework.crawler.jdbc.JdbcCrawler$Craw
> l
> ingProducerThread.run(JdbcCrawler.java:925)
> 
> With a <Grouping>, it stops a bit later:
> 
> jdbc.JdbcCrawler                              - Initializing
> JdbcCrawler...
> jdbc.JdbcCrawler                              - Loaded JDBC driver
> [sun.jdbc.odbc.JdbcOdbcDriver]
> jdbc.JdbcCrawler                              - Connecting to database
> [jdbc:odbc:UsedCars]
> jdbc.JdbcCrawler                              - RowNr 1 of the grouping
> result set is of type [java.lang.Integer], which is derived from 
> [Number]
> jdbc.JdbcCrawler                              - Added final
> GroupingRange with Rangesize 10
> jdbc.JdbcCrawler                              - Closed Grouping
> Resultset
> jdbc.JdbcCrawler                              - Prepared 788 grouping
> ranges based on specified stepping of 13
> jdbc.JdbcCrawler                              - Transforming SQL passed
> from index: [
> 				SELECT ID, Description FROM
> UsedCarsDescriptions where ID BETWEEN %01min AND %01max
> 			]
> jdbc.JdbcCrawler                              - Using transformed SQL
> for PreparedStatement: [
> 				SELECT ID, Description FROM
> UsedCarsDescriptions where ID BETWEEN ? AND ?
> 			]
> jdbc.JdbcCrawler                              - Encountered critical
> Exception in Producer-Thread, storing in class variable
> jdbc.JdbcCrawler                              - DbCrawling terminated
> with Exception
> jdbc.JdbcCrawler                              - Closing JdbcCrawler...
> 
> Any idea anyone?
> 
> Cheers,
> Jürgen
> 
> _______________________________________________
> smila-dev mailing list
> smila-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/smila-dev
> _______________________________________________
> smila-dev mailing list
> smila-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/smila-dev
> _______________________________________________
> smila-dev mailing list
> smila-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/smila-dev
> _______________________________________________
> smila-dev mailing list
> smila-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/smila-dev

Back to the top