[smila-dev] RE: JDBC crawler

[Michael Breidenband <mbreidenband@xxxxxxx>]

Hi Juergen, 

problem no.1 should be fixed - I just commited the fix to svn.

I could not yet reproduce the second problem though. my initial assumption was obviously incorrect: your usage of the grouping element in the index-order you posted seems to be in perfect order (work on the documentation page is in progress). I tuned up the logging-statements a bit: could you run your crawling job again with the current version from the trunk and with the log-level set to TRACE (at least for org.eclipse.smila.connectivity.framework.crawler.jdbc) and send me the log file? alternatively you could provide me with the database schema you used ...

Best regards,
Michael


-----Original Message-----
From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-bounces@xxxxxxxxxxx] On Behalf Of Michael Breidenband
Sent: Dienstag, 25. November 2008 09:26
To: Smila project developer mailing list
Subject: [smila-dev] RE: JDBC crawler

hi, 

the first problem is very likely a bug (Mea culpa). I shall check this and provide a fix ASAP. 

the second problem seems to be related to improper use of the <Grouping> element - very likely caused by the veritable lack of documentation about the format of the IndexOrder. I shall provide a wiki page - likewise ASAP. could you post the IndexOrder you were using? 


Kind regards,
Michael


-----Original Message-----
From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-bounces@xxxxxxxxxxx] On Behalf Of August Georg Schmidt
Sent: Montag, 24. November 2008 15:21
To: Smila project developer mailing list
Subject: [smila-dev] RE: JDBC crawler

Michael is currently ill. I'll notify him when he is back. This should be an error.

-----Original Message-----
From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-bounces@xxxxxxxxxxx] On Behalf Of Juergen.Schumacher@xxxxxxxxxxx
Sent: Montag, 24. November 2008 15:19
To: smila-dev@xxxxxxxxxxx
Subject: [smila-dev] JDBC crawler

Hi,

I've tried to use the new JDBC crawler in SMILA, but I did not succeed yet. 
Is there any documentation about the IndexOrder format? I did not find something in the Wiki. 
After fixing the DS declarations to get the crawler service registered to the CrawlerController 
(have committed it), I'm still running into one of two problems:
Without a <Grouping> in the <Selections> element, I get

jdbc.JdbcCrawler                              - Encountered RuntimeException in ProducerThread
java.lang.NullPointerException
	at org.eclipse.smila.connectivity.framework.crawler.jdbc.JdbcCrawler.populateRetrievalResultSet(JdbcCrawler.java:624)
	at org.eclipse.smila.connectivity.framework.crawler.jdbc.JdbcCrawler.hasNext(JdbcCrawler.java:492)
	at org.eclipse.smila.connectivity.framework.crawler.jdbc.JdbcCrawler.access$7(JdbcCrawler.java:487)
	at org.eclipse.smila.connectivity.framework.crawler.jdbc.JdbcCrawler$CrawlingProducerThread.run(JdbcCrawler.java:925)

With a <Grouping>, it stops a bit later:

jdbc.JdbcCrawler                              - Initializing JdbcCrawler...
jdbc.JdbcCrawler                              - Loaded JDBC driver [sun.jdbc.odbc.JdbcOdbcDriver]
jdbc.JdbcCrawler                              - Connecting to database [jdbc:odbc:UsedCars]
jdbc.JdbcCrawler                              - RowNr 1 of the grouping result set is of type [java.lang.Integer], which is derived from [Number]
jdbc.JdbcCrawler                              - Added final GroupingRange with Rangesize 10
jdbc.JdbcCrawler                              - Closed Grouping Resultset
jdbc.JdbcCrawler                              - Prepared 788 grouping ranges based on specified stepping of 13
jdbc.JdbcCrawler                              - Transforming SQL passed from index: [
				SELECT ID, Description FROM UsedCarsDescriptions where ID BETWEEN %01min AND %01max
			]
jdbc.JdbcCrawler                              - Using transformed SQL for PreparedStatement: [
				SELECT ID, Description FROM UsedCarsDescriptions where ID BETWEEN ? AND ?
			]
jdbc.JdbcCrawler                              - Encountered critical Exception in Producer-Thread, storing in class variable
jdbc.JdbcCrawler                              - DbCrawling terminated with Exception
jdbc.JdbcCrawler                              - Closing JdbcCrawler...

Any idea anyone?

Cheers,
Jürgen 

_______________________________________________
smila-dev mailing list
smila-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/smila-dev
_______________________________________________
smila-dev mailing list
smila-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/smila-dev
_______________________________________________
smila-dev mailing list
smila-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/smila-dev