News related to PubCrawler and its WWW-Service

Join the PubCrawler Mailing List to receive new entries by e-mail!

14/03/07 Missing e-mail notifications
  Due to a malfunctioning script some of the e-mail notifications and results were not sent out over the last six weeks.

If you didn't get a results update you can recover hits from a certain time interval with the Quickstart+ feature at http://pubcrawler.gen.tcd.ie/quickstart2.html.
Simply fill in your PubCrawler user name and password, the start and end date to search for and click on 'Quickstart'.

Apologies for any inconvenience caused!


11/12/06 Service Outage
  Due to power interruptions throughout the university, the PubCrawler service was unavailable from Friday evening, 8th December, until Monday morning, 11th December.

If you didn't get a results update you can either wait for the next scheduled run or you can start your queries manually with the Quickstart feature at http://pubcrawler.gen.tcd.ie/quickstart.html.
Alternatively, you can log on to the main Web portal at http://pubcrawler.gen.tcd.ie and then click the button 'Start queries!' near the top of the page.

Apologies for any inconvenience caused!

Update: We got hit by another power cut on Monday, 18th December, and this time one of our servers didn't survive it. As of today (Friday, 22nd December) the system is running fine and the service should be back to normal.

30/09/05 Missing First Authors
  Due to a programming bug the PubCrawler results generated over the last week (Sep 22rd, 20:33 till Sep 30th, 01:20 IST) were corrupted, i.e. the first author name was never listed.

To get the correct view of your results over that period, please go to the Quickstart+ page at http://pubcrawler.gen.tcd.ie/quickstart2.html
Fill in your PubCrawler user name and password,
specify all necessary e-mail and formating options,
insert a starting date and click on 'Quickstart'.
New results will be available in a short moment.

Apologies for any inconvenience caused!

06/07/05 Service Outage
  Due to an unfortunate combination of network problems and absence of the good people who normally help out in such cases, the PubCrawler service was unavailable over the last three days.
Full functionality has now been restored and the next results will bring you up-to-date with the latest publications in your field of research.
In some cases notifications of new hits found on Monday could not be sent out in time, so please make sure to look for older hits in your results page, located at http://pubcrawler.gen.tcd.ie/db/username or accessible after login through the entry portal at http://pubcrawler.gen.tcd.ie

Apologies for any inconvenience caused!

18/06/05 Service Outage
  Due to a power outage PubCrawler was down over the weekend and only a very limited number of user queries have been executed between Saturday, June 18th and Monday, June 20th.
If you didn't get a results update, you can either wait for the next scheduled run or you can start your queries manually with the Quickstart feature at http://pubcrawler.gen.tcd.ie/quickstart.html.
Alternatively, you can log on to the new web-interface at http://pubcrawler.gen.tcd.ie and then click the button 'Start queries!' near the top of the page.

Apologies for any inconvenience caused!

02/05/04 New output format
  From today on, all PubCrawler users will receive their results in a new format, designed by Indra Konnerth (see example). This has been in place already for users who registered after February 15th.
The reason for the update is the requirement by NCBI to use their E-Utilities for querying the databases. Incorporating newly retrieved results needs some formating and we used this opportunity to introduce a new output format that differs slightly from the NCBI look and feel, but is easier to maintain (i.e., more resistant to changes at NCBI). It also helps overcoming incompatibility problems with certain mail clients and avoidance of accidental classification as spam by allowing a more flexible choice of options for the mailed results. This will become fully usable with an update of the PubCrawler website, which is currently under development. You can try out a preliminary version at pubcrawler.gen.tcd.ie/index_new.html.
If you encounter any problems with the new output format (or the new website), please report them to pubcrawlerREMOVECAPShelp@gmail.com.

22/09/03 GenBank Queries
  Due to changes on the Entrez site PubCrawler could not retrieve any GenBank hits for the last few weeks. Instead, an error message was shown. The program has now been patched up to fix this and those of you with GenBank queries should get updates again.
The malfunction of the GenBank queries has first been reported on August 16th (thanks to John and everybody else who mailed afterwards!). By default, PubCrawler only reports items that have been modified/inserted within the last 30 days. Unless you have changed the relentrezdate-option to 50 days (the next higher value) or more, you might miss out on some new entries. To retrieve these, please use the Quickstart+ feature that allows you to carry out your searches on a specified data range (see PubCrawler FAQ, article 4b for more info).

26/05/03 Service Outage
  Due to network problems beyond our control, PubCrawler was cut-off from the internet over the weekend and only a very limited number of user queries have been executed between Saturday, May 24th and Monday, May 26th. If you didn't get a results update, you can either wait for the next scheduled run or you can start your queries manually with the Quickstart feature at http://pubcrawler.gen.tcd.ie/quickstart.html.

Apologies for any inconvenience caused!

21/01/03 Server Outage
  The PubCrawler server crashed due to NFS problems and was unavailable from Tuesday evening until Wednesday morning. To comply with time restrictions for searches at NCBI only those jobs that are explicitly scheduled for Wednesday were carried out. If you prefer not to wait for the next scheduled execution of your queries, please use the Quickstart feature at to start them instantly.

Apologies for any inconvenience caused!

26/08/02 Server Outage
  The PubCrawler server crashed at the start of the weekend and couldn't be restarted until Monday morning. This affected all of the user jobs scheduled for Sunday and some of the Saturday and the Monday ones. If you prefer not to wait for the next scheduled execution of your queries, please use the Quickstart feature at to start them instantly.

Apologies for any inconvenience caused!

27/06/02 New FAQ
  A list of frequently asked questions (and answers) has been compiled for PubCrawler's WWW-service and can be found at http://pubcrawler.gen.tcd.ie/pubcrawler_www_faq.html

07/06/02 Search Fields Update
  PubCrawler Configurator has been updated to allow specification of all the search field tags that are defined for PubMed.
The new ones include 'Personal Name as Subject', 'Secondary Source ID', and 'Title/Abstract'.

Description for all search fields can be found at the PubMed help page: http://www.ncbi.nlm.nih.gov/entrez/query/static/help/pmhelp.html#SearchFieldDescriptionsandTags

Please note that for compatibility reason the 'MEDLINE ID' field is still available, but it's use is deprecated because these identifiers are no longer assigned to PubMed citations. Please use PubMed ID's instead.

12/04/02 New URLs
  PubCrawler has a new address: All previous links starting with
www.gen.tcd.ie/pubcrawler/...
should now be
pubcrawler.gen.tcd.ie/...

Results pages will still be accessible through the old links but everything else is automatically redirected to the new location.

Apologies for the lack of advanced warning, but these changes suddenly had to be rushed through.

If you notice any problems, please write an e-mail to pubcrawlerREMOVECAPShelp@gmail.com with a detailed description.

28/12/01 Previous Results
  For those of you who missed a few PubCrawler results due to holidays, etc., there is now a modified Quickstart which allows you to retrieve all results with relentrezdate/relmoddate within a certain range.

Just go to your PubCrawler results page at http://pubcrawler.gen.tcd.ie/db/username (or http://pubcrawler.gen.tcd.ie/cgi-bin/pubcrawler.results if 'Easy Check' is activated) and click on the 'Previous Results' button on the top right of the page. It will bring up a form in which you can specify a date range and start the immediate execution of your queries.

This is only a temporary hack and will be improved soon!

All the best for 2002 and Happy PubCrawling!

13/12/01 Service interruption
  On Sunday 16th December 2001 between the hours of 0700-1900 (GMT), there will be no access to PubCrawler due to college maintenance work.
We will attempt to carry out all user jobs specifically scheduled for Sunday before that time.
The service should be back to normal on Monday.

Apologies for any inconvenience cause.

p.s.: Please remember that you can suspend your scheduled queries, which might be handy for the upcoming holiday season: http://pubcrawler.gen.tcd.ie/suspend_help.html


02/08/01 Restart of Service
  The PubCrawler Service is running again, but further disruptions are possible aproximately until the end of August.

18/07/01 Suspension of Scheduled Queries
  For the next two weeks all scheduled PubCrawler jobs, which are normally run automatically on a number of servers, will be suspended. Due to renovation work only the main server is running, which wouldn't be able to take the load. However, if you would like to update your results you can use Quickstart to start your PubCrawler jobs manually.
At peak times connection to the server might be slow. If you can't access the website, please try again later.

As soon as circumstances allow it (probably within two to four weeks) the service will be back as usual. Please bear with us!

27/06/01 Possible Disruptions
  Due to renovation work in our offices, which will also affect electricity and network cabling, disruptions to PubCrawler's WWW-service are expected until the middle of August.
If you can't access the website, please try again later.
If your queries were not carried out or got stuck, please use Quickstart to start your PubCrawler jobs manually.

Apologies for any inconvenience this might cause!

07/05/01 Faulty PubMed Records
  PubCrawler reported hits that contained journal names like 'jid9310916'. The link from the author name produces an error message.

Explanation: PubMed experienced some problem which resulted in the reports mentioned above. The faulty entries were withdrawn and will be added again in the future.
It's not certain that the same hits (with corrected journal names) will show up again in the PubCrawler results page. It is therefore recommended to check the 'x-day-old records'-links in your PubCrawler results until they come up with the proper entries.

Apologies for any inconvenience caused!

26/01/01 No New Hits
  Since January 18th PubCrawler did not report any new hits for existing PubMed queries. This is not a fault of the program. It rather seems as if the PubMed database has not been updated for the last week. We have sent queries regarding this to NCBI but did not get a response so far. Hopefully this will be sorted out soon.


Update: http://www.nlm.nih.gov/pubs/techbull/jf01/jf01_technote.html#pubmed
(Thanks to Miriam for passing on this information!)

Update: The first chunk of new articles appeared: 6054 new articles added on 2001/02/07. These will all be automatically scanned by PubCrawler, given that your next searches will be carried out within the number of days specified by the 'relentrezdate' variable within your PubCrawler options (default: 30 days). If your queries are carried out at least once a week there should be no need to change your settings.

13/12/00 No Automated Query Runs
  Due to network problems during the night the main server overloaded and couldn't start the queries scheduled for today. The service is back to normal now.
If you would like updates on your queries today, please use Quickstart to run your PubCrawler jobs manually.

Apologies for any inconvenience caused!

14/11/00 Problems with Neighbourhood Searches
  Changes on the NCBI site interrupted PubCrawler's Neighbourhood Search facility. The service for related documents in PubMed could be restored but searching for related sequences in GenBank is currently not possible through PubCrawler and has been disabled.
It's not certain if this feature will be available again in the future - we are awaiting information from NCBI regarding this. In the meantime we recommend to use the following tools for sequence updates:
  • Peer Bork's lab at EMBL (BLAST sequence searches against DNA and protein databases)
  • Swiss-Shop at Expasy (BLAST sequence and keyword searches against SwissProt updates)
Please note: Medline Unique Identifiers (UIs) don't work anymore for related articles. If you notice strange or no results your search might be affected and you need to change the identifier to the corresponding PubMed Identifier (PMID). See Neighbour Help for details.
If you have any problems to report, please write to pubcrawlerREMOVECAPShelp@gmail.com.

We regret any inconvenience caused by this!

13/08/00 Hardware Problems
  A complete harddrive failure caused quite some chaos over the weekend. The disk has been replaced and we hope that the PubCrawler WWW-service will run stable now.
Some data might be corrupted or lost. In that case try Quickstart to run your PubCrawler jobs manually. Also don't forget to check the links for hits from previous days.
If you have any problems to report, please write to pubcrawlerREMOVECAPShelp@gmail.com.

We are very sorry for any inconvenience caused by this!

31/07/00 Service Interruption
  The PubCrawler server needs to shut down on Wednesday, August 2, 7.00 pm GMT for some hardware upgrade.
The increasing number of queries that need to be carried out every day has brought the server very close to the limit of its memory capacity. After 120 days of uptime it will therefore shut down for the necessary work. The service will hopefully be back online one or two hours later.
The automated queries will not be affected by this!

20/07/00 5000 Users
  We are happy to announce that today the 5000th user has joined the PubCrawler WWW-Service!





The following table lists the email domains of our currently active users (= at least one query set up):
#DOMDomainActive UsersPercent
1  EDUUS Educational94418.8 %
2  COMUS Commercial72514.5 %
3  NLNetherlands4809.6 %
4  UKUnited Kingdom4659.3 %
5  FRFrance3226.4 %
6  CACanada3146.3 %
7  DEGermany2895.8 %
8  IEIreland2334.6 %
9  AUAustralia1593.2 %
10  NETNetwork961.9 %
11  ORGNon-Profit Organization941.9 %
12  BEBelgium921.8 %
13  ESSpain821.6 %
14  JPJapan811.6 %
15  ITItaly631.3 %
16  DKDenmark581.2 %
17  GOVUS Government571.1 %
18  other 4639.2 %
Total56 domains5017 users


Sláinte!

06/05/00 4000 Users
  The number of PubCrawler users keeps growing! Shortly after a year of operating the WWW-Service we have surpassed the 4000 mark:




The following table lists the email domains of our currently active users (= at least one query set up):
 DOMDomainUsersPercent
1  EDUUS Educational79019.6 %
2  COMUS Commercial54213.5 %
3  NLNetherlands3969.8 %
4  UKUnited Kingdom3518.7 %
5  CACanada2796.9 %
6  FRFrance2476.1 %
7  DEGermany2295.7 %
8  IEIreland2185.4 %
9  AUAustralia952.4 %
10  NETNetwork852.1 %
11  ORGNon-Profit Organization792.0 %
12  JPJapan661.6 %
13  BEBelgium661.6 %
14  ESSpain621.5 %
15  BRBrazil501.2 %
16  ITItaly481.2 %
17  NONorway441.1 %
18  other 3759.3 %
Total55 domains4022 users

Sláinte!

13/04/00 Results per Email Improved
  After the format change at NCBI some users experienced problems with results sent to them by email. We have introduced two measures which will hopefully help:
  • A patch was applied to PubCrawler which undecodes the ampersand sign (&) in hypertext links.
    Thanks very much to Dmitry Mozzherin from BioMail for the hint!

  • A new option for the emailed results is available:
    In the slim HTML page general NCBI links and tables are filtered out.

    Please modify your settings (following the according button from the top of your results page) if you like to change the format of results emailed to you.
If you continue to experience problems with this, please send an email to Karsten at pubcrawlerREMOVECAPShelp@gmail.com with a detailed description, including mail client and operating system.

25/03/00 Huge Number of Hits
  Shortly after applying a local patch to the PubCrawler script, which temporarily solved the problem of old hits (see 18/03/00), this was foiled again by sudden changes at the NCBI site.
Many users whose queries were executed on Friday (24/03/00) and Saturday (25/03/00) will therefore find a huge increase in hits which include a lot of backdated reports.
We have changed PubCrawler's internals again to deal with this problem. It's currently running normal - until the next surprise from NCBI!
You can filter out all the old hits from your results by starting your queries again manually:
  1. go to PubCrawler's Quickstart
  2. fill in your username and password
  3. click the 'Quickstart'-button
  4. follow the link to your results page
  5. click the refresh/reload button of that browser and continue with that until searches are terminated
If you experience any problems with this, please send an email to Karsten at pubcrawlerREMOVECAPShelp@gmail.com .

24/03/00 No More No-Hit Emails
  From tomorrow no more emails will be sent out on days when PubCrawler does not retrieve any new hits. This will affect both, notifications and full results emails.
However, if you prefer to get emails every time your searches are carried out, then you have to change your settings accordingly:
  1. go to your results page ( http://pubcrawler.gen.tcd.ie/db/user_name )
  2. click on the button labeled 'Modify Settings'
  3. you will reach a page with all your settings (either directly or after password dialog)
  4. find a green box starting with 'Do you wish to get emails....'
  5. click the button beside 'send mail every time'
  6. fill in your password and click on 'Change Data'
Please note: Every day a lot of emails bounce back because of network difficulties and routing problems, which are beyond our control. Please don't rely solely on mails/notifications from PubCrawler! You should check your results on the web regularly to make sure you don't miss any new hits.

Check out older news as well!




[Home] [PubCrawler WWW-Service]  [PubCrawler Configurator]  [PubCrawler Program]



Last modified at $Date: 2006/12/22 14:00:13 $
contact: pubcrawlerREMOVECAPShelp@gmail.com