[MKSearch-dev] Jspider postgres problems

[Chris Croome]

Hi

On Wed 25-Jan-2006 at 11:04:40AM -0000, Phil Shaw wrote:
> On 23 Jan 2006 at 15:56, Chris Croome wrote:
> 
> > Earlier today I did have jspider adding triples to a mksearch_test
> > database, but then I dropped the db and recreated it and now I can't
> > get this working again... the test scripts run fine
> > (java-pgsql-test.sh and
> 
> Snip
> 
> > [Engine] starting with configuration 'rdfstoredb'
> 
> This line shows you're using the right JSpider configuration set.
> 
> > Loading 2 plugins.
> 
> Snip
> 
> > Error casting to StoreManager. Check your configuration file. Could
> > not create repository. Unable to load JDBC-driver
> > 'org.postgresql.Driver' plugin instantiated.
> 
> This line shows that the postgres database driver is not in your
> classpath. I suspect you are using the non-database JSpider script:
> 
> $mk_home/bin/java-jspider.sh
> 
> You should use one of the scripts that includes the database driver in
> the classpath, e.g.
> 
> $mk_home/bin/java-jspider-pgsql.sh http://www.test.example/ rdfstoredb

Right, this was the problem :-)

> It's okay to have redundant database driver JARs in the classpath, so
> the standard start-up script could be amended to include PostgreSQL
> and MySQL drivers. That would make things simpler and perhaps avoid
> this sort of mis-match.

OK, that is worth bearing in mind.

I have just tried to index 3 sites using a simple script, before I ran
this I dropped and recreated the database:

  dropdb mksearch_test; createdb mksearch_test

This is the script:

  #!/bin/bash
  SITES="tre.ngfl.gov.uk ferl.becta.org.uk www.aclearn.net"
  for a in $SITES
    do
    $mk_home/bin/java-jspider-pgsql.sh http://$a/ rdfstoredb
  done

And it runs for a while with various warnings and then ends with this:

  PANIC! Task net.javacoding.jspider.core.task.work.SpiderHttpURLTask at 1231fd8 threw an excpetion!  java.lang.NullPointerException

And when I look at the database to see what is there I get:

  mksearch_test=> select * from literals;
   id | datatype |      labelhash       | language |                                                                                                                                    label                                                                                                                                    
  ----+----------+----------------------+----------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
   -1 |        0 | -7566735784996357570 |          | Community Learning Resource
   -2 |        0 | -1621285313438006658 |          | 
   -3 |        0 |  8274037457894185038 |          | The Community Learning Resource website supports the Adult and Community Learning (ACL) sector. It provides information, advice and guidance to those working in the sector and is designed to compliment the rollout of effective e-learning and related support into ACL.
  (3 rows)

Any ideas what migh be the problem? Is there any debugging information
that I could collect and send to this list?

Chris

-- 
Chris Croome                               <chris at webarchitects.co.uk>
web design                             http://www.webarchitects.co.uk/ 
web content management                               http://mkdoc.com/