[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
RE: [smila-dev] RE: JDBC crawler
|
Hi,
yes, it works now. Thanks (:
Cheers,
Jürgen
> -----Original Message-----
> From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-
> bounces@xxxxxxxxxxx] On Behalf Of Michael Breidenband
> Sent: Thursday, November 27, 2008 5:10 PM
> To: Smila project developer mailing list
> Subject: RE: [smila-dev] RE: JDBC crawler
>
> Hi Juergen,
>
> the JdbcOdbc-Bridge seems to choke on leading whitespaces and/or
> linebreaks in the SQL statement:
>
> <SQL>SELECT ID, Description FROM UsedCarsDescriptions</SQL>
>
> works fine whereas
>
> <SQL>
> SELECT ID, Description FROM UsedCarsDescriptions
> </SQL>
>
> produces an empty resultset (which results in zero records being
> created, of course). The Derby driver does not seem to have this
> problem: it runs fine with both versions.
>
> The SQL is stripped of leading whitespaces now and I got about 10k
> records from your database. I added some Testcases for JdbcOdbcCrawling
> to the Test-Bundle as well, but the are not active (outcommented) as
> they would probably break the build with the data source not being
> present on Bamboo. They should work on your machine though.
>
> Kind regards,
> Michael
>
>
>
> -----Original Message-----
> From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-
> bounces@xxxxxxxxxxx] On Behalf Of Juergen.Schumacher@xxxxxxxxxxx
> Sent: Donnerstag, 27. November 2008 10:55
> To: smila-dev@xxxxxxxxxxx
> Subject: RE: [smila-dev] RE: JDBC crawler
>
> Hi Michael,
>
> here you are: two logs with TRACE for the JDBC crawler.
> one with grouping enabled and one without grouping.
> There is a nice exception in the grouping run, hope it helps. I can
> send you the database file off-list if you need it.
> The run without goruping doesn't seem to produce an error, but it also
> does not produce records, it seems (-;
>
> Cheers,
> Jürgen.
>
>
> > -----Original Message-----
> > From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-
> > bounces@xxxxxxxxxxx] On Behalf Of Michael Breidenband
> > Sent: Wednesday, November 26, 2008 3:09 PM
> > To: Smila project developer mailing list
> > Subject: [smila-dev] RE: JDBC crawler
> >
> > Hi Juergen,
> >
> > problem no.1 should be fixed - I just commited the fix to svn.
> >
> > I could not yet reproduce the second problem though. my initial
> > assumption was obviously incorrect: your usage of the grouping
> element
> > in the index-order you posted seems to be in perfect order (work on
> > the documentation page is in progress). I tuned up the
> > logging-statements a
> > bit: could you run your crawling job again with the current version
> > from the trunk and with the log-level set to TRACE (at least for
> > org.eclipse.smila.connectivity.framework.crawler.jdbc) and send me
> the
> > log file? alternatively you could provide me with the database schema
> > you used ...
> >
> > Best regards,
> > Michael
> >
> >
> > -----Original Message-----
> > From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-
> > bounces@xxxxxxxxxxx] On Behalf Of Michael Breidenband
> > Sent: Dienstag, 25. November 2008 09:26
> > To: Smila project developer mailing list
> > Subject: [smila-dev] RE: JDBC crawler
> >
> > hi,
> >
> > the first problem is very likely a bug (Mea culpa). I shall check
> this
> > and provide a fix ASAP.
> >
> > the second problem seems to be related to improper use of the
> > <Grouping> element - very likely caused by the veritable lack of
> > documentation about the format of the IndexOrder. I shall provide a
> > wiki page - likewise ASAP. could you post the IndexOrder you were
> > using?
> >
> >
> > Kind regards,
> > Michael
> >
> >
> > -----Original Message-----
> > From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-
> > bounces@xxxxxxxxxxx] On Behalf Of August Georg Schmidt
> > Sent: Montag, 24. November 2008 15:21
> > To: Smila project developer mailing list
> > Subject: [smila-dev] RE: JDBC crawler
> >
> > Michael is currently ill. I'll notify him when he is back. This
> should
> > be an error.
> >
> > -----Original Message-----
> > From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-
> > bounces@xxxxxxxxxxx] On Behalf Of Juergen.Schumacher@xxxxxxxxxxx
> > Sent: Montag, 24. November 2008 15:19
> > To: smila-dev@xxxxxxxxxxx
> > Subject: [smila-dev] JDBC crawler
> >
> > Hi,
> >
> > I've tried to use the new JDBC crawler in SMILA, but I did not
> succeed
> > yet.
> > Is there any documentation about the IndexOrder format? I did not
> find
> > something in the Wiki.
> > After fixing the DS declarations to get the crawler service
> registered
> > to the CrawlerController (have committed it), I'm still running into
> > one of two problems:
> > Without a <Grouping> in the <Selections> element, I get
> >
> > jdbc.JdbcCrawler - Encountered
> > RuntimeException in ProducerThread
> > java.lang.NullPointerException
> > at
> >
> org.eclipse.smila.connectivity.framework.crawler.jdbc.JdbcCrawler.popu
> > l
> > ateRetrievalResultSet(JdbcCrawler.java:624)
> > at
> >
> org.eclipse.smila.connectivity.framework.crawler.jdbc.JdbcCrawler.hasN
> > e
> > xt(JdbcCrawler.java:492)
> > at
> >
> org.eclipse.smila.connectivity.framework.crawler.jdbc.JdbcCrawler.acce
> > s
> > s$7(JdbcCrawler.java:487)
> > at
> >
> org.eclipse.smila.connectivity.framework.crawler.jdbc.JdbcCrawler$Craw
> > l
> > ingProducerThread.run(JdbcCrawler.java:925)
> >
> > With a <Grouping>, it stops a bit later:
> >
> > jdbc.JdbcCrawler - Initializing
> > JdbcCrawler...
> > jdbc.JdbcCrawler - Loaded JDBC driver
> > [sun.jdbc.odbc.JdbcOdbcDriver]
> > jdbc.JdbcCrawler - Connecting to
> database
> > [jdbc:odbc:UsedCars]
> > jdbc.JdbcCrawler - RowNr 1 of the
> grouping
> > result set is of type [java.lang.Integer], which is derived from
> > [Number]
> > jdbc.JdbcCrawler - Added final
> > GroupingRange with Rangesize 10
> > jdbc.JdbcCrawler - Closed Grouping
> > Resultset
> > jdbc.JdbcCrawler - Prepared 788 grouping
> > ranges based on specified stepping of 13
> > jdbc.JdbcCrawler - Transforming SQL
> passed
> > from index: [
> > SELECT ID, Description FROM
> > UsedCarsDescriptions where ID BETWEEN %01min AND %01max
> > ]
> > jdbc.JdbcCrawler - Using transformed SQL
> > for PreparedStatement: [
> > SELECT ID, Description FROM
> > UsedCarsDescriptions where ID BETWEEN ? AND ?
> > ]
> > jdbc.JdbcCrawler - Encountered critical
> > Exception in Producer-Thread, storing in class variable
> > jdbc.JdbcCrawler - DbCrawling terminated
> > with Exception
> > jdbc.JdbcCrawler - Closing
> JdbcCrawler...
> >
> > Any idea anyone?
> >
> > Cheers,
> > Jürgen
> >
> > _______________________________________________
> > smila-dev mailing list
> > smila-dev@xxxxxxxxxxx
> > https://dev.eclipse.org/mailman/listinfo/smila-dev
> > _______________________________________________
> > smila-dev mailing list
> > smila-dev@xxxxxxxxxxx
> > https://dev.eclipse.org/mailman/listinfo/smila-dev
> > _______________________________________________
> > smila-dev mailing list
> > smila-dev@xxxxxxxxxxx
> > https://dev.eclipse.org/mailman/listinfo/smila-dev
> > _______________________________________________
> > smila-dev mailing list
> > smila-dev@xxxxxxxxxxx
> > https://dev.eclipse.org/mailman/listinfo/smila-dev
> _______________________________________________
> smila-dev mailing list
> smila-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/smila-dev
