Faster Newsgroup Access With Hamster

by Dave Harms

Published 2007-02-22    Printer-friendly version

Advertisement

IN PRINT

Clarion Tips & Techniques Vol 4

Topics include: browses and forms; embeds; templates; threading issues; controlling printers; SQL; Vista; managing source code with version control; creating DLLs; calculating dates; using finite state machines; and much more.

Buy Now!

Most of the time I find the SoftVelocity newsgroups painfully slow. I don't mean the people are dense (although that happens often enough in chat), I mean it can take the better part of a minute for a message to load after I click on the header in Thunderbird. I don't know if that's a Thunderbird problem or a server problem, but I know I'm not alone. Ordinarily I wouldn't even notice as I prefer to set my newsreader to download all messages automatically, but Thunderbird, which I otherwise like a lot, has a "download all" feature that, for my purposes, is essentially useless and buggy to boot.

A few days ago Sean Cameron posted a message (which I eventually managed to read) saying that CapeSoft uses Hamster, a news and mail proxy server for Windows, to get around this problem.

I'm indebted to Sean. Since installing Hamster my newsgroup reading experience has been transformed. Now that I have a local cache of the SV server I never have to wait for a message to load.

There were, however, a few bumps on the road to Nirvana. In this article I'll show you how to set up Hamster and configure it to cache the SV news server.

Installing Hamster

The first thing is to make sure you're using the right version of Hamster. I started out with something called Hamster Playground from www.elbiah.de/hamster. I'm not going to make that link live, because I don't want you to go there accidentally. No, it isn't a porn site. Hamster Playground is in fact Hamster, but it appears to be some early version and it has an unbelievably convoluted user interface (not that I find the Hamster I use that much more intuitive, but anyway...). The main problem I had with Hamster Playground, once I figured out how to set it up, was that I couldn't get it to post to the SV news server. Instead I had to post all messages from Thunderbird directly to the SV news server, which was a bit of a hassle. Then I asked Jono Woodhouse how he'd set up posting to the SV server, and as soon as I got his screen shots I knew I had the wrong rodent in the house. Thanks, Jono!

The version you and I want is Hamster Classic 2.0 or later. I downloaded 2.1.0.11 from SoftPedia.

Installing Hamster is a breeze. It comes as a zip file, and you unpack it into whatever directory you like. A tip for those of you who travel: I occasionally sync my Thunderbird setup to my laptop, and to keep this process simple I put Hamster in a subdirectory of my Thunderbird data folder. You can install Hamster on your desktop machine or on another machine on your network; it really doesn't matter as long as you can access Hamster's port, which defaults to 119, the standard NNTP port.

Hamster comes with two executables: hamster.exe and ham.exe. Hamster.exe is the one you want to run, and you'll probably want to set up a shortcut as well.

The first time you run Hamster (see Figure 1) you'll see a bunch of warning messages about directories Hamster needed to create. You can safely ignore these warnings.

Figure 1. Hamster on startup

Next you need to set the admin password. As noted in the help, you won't be able to connect to Hamster with your newsreader unless you do this. Choose Configuration, Accounts & Passwords ..., highlight the second line (Hamster Administrator) and click on Edit. You'll see the window shown in Figure 2.

Figure 2. Changing the admin password

Click on Change to set a new password. You need this for your news server to access Hamster (and you'll need to tell your newsreader to use the userid admin and the password you specified).

Now go back to the main menu to create a news server record. Choose Configuration|News: Server, Groups, Pulls .... for the News Configuration window (see Figure 3).

Figure 3. Setting the pull limits

On the General tab the default settings, at least on the version I installed, are to pull only the 100 oldest items for each group. That makes no sense on the SV groups; change this to the newest items. I'm pulling the 1000 newest.

Now to go the Newsserver tab and click on Add. Enter the SV news server name as shown in Figure 4.

Figure 4. Setting the server name

Click OK and the server settings window appears (Figure 5).

Figure 5. Server settings

If you're using a userid/password with the SV news server click on Edit to set those values.

At this point you've configured the news server but you don't yet have a list of newsgroups. To get those you have to go online. Go back to the main Hamster screen and choose Online|All NNTP-servers (or you can specify just the SV news server you set up). This tells Hamster to go online and do any waiting tasks (and like a good little rodent it knows that you need the list of newsgroups because you've just added the server).

Now you need to tell Hamster which of the newsgroups you want to pull from the SV server. Go to Configuration|News: Server, Groups, Pulls ..., make sure the SV news server is highlighted, and click on News-Pulls. Select the newsgroups you want to download and click on Subscribe selected groups (see Figure 6). Note the Show groups with drop list. The available options include matching on a regular expression, simple string matching, "begins with", and a really nifty one for anyone who uses newsgroups a lot, match on initials. For this last option "clc" would match comp.lang.clarion.

Figure 6. Selecting the newsgroups

With the groups selected you're now ready to download messages. Go to the main menu and once again choose Online| All NNTP-Servers. Depending on the Pull-limit options you specified earlier it may take a long time to download messages and get the database up to date.

To update the local cache you can do one of two things - repeat the last step of going online, which gets a bit tedious after a while, or set up an automated task to poll the SV news server.

Periodic news retrieval

If you want to automatically retrieve newsgroup messages at regular intervals you need to create a simple script. Choose Script|Manage scripts and modules ..., then click on the Scripts tag (Figure 7).

Figure 7. Managing scripts

Click on New and enter a suitable file name (Figure 8). You don't need to specify the .hsc extension.

Figure 8. Creating a script

When you click OK Hamster opens notepad with a default script containing a comment and a quit statement. Delete these and paste in the following text:

#!hs2
 while( true )
  HamNewsJobsClear
  HamNewsJobsPostDef
  HamNewsJobsPullDef
  HamNewsJobsStart
  HamWaitIdle
  sleep(300000 )
  endwhile
quit

This script is an endless loop which posts and pulls newsgroup messages, waits until all tasks are done, and then sleeps for five minutes (sleep times are in milliseconds) before looping again.

Save the file and exit notepad; you should now see GetNews.hsc listed in the scripts list. Go back to the main menu.

To run the script immediately just select it from the Script menu where you'll now see it listed.

Running Hamster automatically

There are two more parts to fully automating Hamster: starting Hamster automatically, and starting the script automatically.

To run Hamster at startup just create a shortcut to Hamster.exe and copy it to your \Start Menu\Programs\Startup folder (at least under XP - I haven't tried this with Vista yet).

To tell Hamster to run the script automatically on startup choose Configuration|Automation and click on the Actions tab (Figure 9). Expand Hamster and click on Startup. In the Run script dropcombo choose GetNews.hsc.

Figure 9. Setting the script to run at startup

Now you're all set. Hamster will run whenever you start Windows, and it will poll the SV news server for news items at the interval you specify in the script.

Hamster minimizes to the system tray so it's out of the way unless you need it. Sean Cameron says he's found Hamster to be a bit unstable and in occasional need of a restart. I haven't noticed any problems yet; your mileage may vary.

Posting messages

Unlike my experience with Hamster Playground, I didn't encounter any problems posting messages back to the SV server using Hamster Classic. I didn't have to modify any other settings; Hamster simply accepts my messages and posts them to the SV server on the next iteration of the script.

However, as Jono Woodhouse has pointed out to me, there is a potential problem with using Hamster as a proxy for multiple users. Some of SV's groups are public access, and some require a password. If you're logging in, the server enforces the name that's in your login profile on the SV server. You can't change that simply by changing your newsgroup client settings. Because of this, all messages posted to the SV news server via a login account will appear to come from one user. For this reason Jono has set up two news servers within Hamster, one for the public access newsgroups, and another for the subscriber-only newsgroups.

Summary

If like me you find the SoftVelocity newsgroups too slow, give Hamster a try. It's free, it's fairly easy to configure (once you know what to do) and it will give you lightning-fast access to newsgroup messages. Hamster even supports NNTP searches, something missing from the SV server.

Hamster also offers mail proxying (POP3, SMTP, and IMAP) if that's something you're after.

Download Hamster


David Harms is an independent software developer and the editor and publisher of Clarion Magazine. He is also co-author with Ross Santos of Developing Clarion for Windows Applications, published by SAMS (1995), and has written or co-written several Java books. David is a member of the American Society of Journalists and Authors (ASJA).

Printer-friendly version

Reader Comments

Posted on Monday, February 26, 2007 by Dave Harms

I've updated the text with a correction - Jono pointed out to me that he's set up two news servers within Hamster, not two Hamster servers, as might have been construed from the text.

Dave

 

Posted on Wednesday, February 28, 2007 by Phillip Carroll

Well, it certainly sounded like a great idea.  I guess "your mileage may vary" applies here.

Unfortunately, now the messages in TBird no longer match the headers.  In fact, I have fewer messages than headers.  Things are really scrambled up!  

In addition, I now get from TBird:  
Error!
newsgroup server responded:no such article number in this group

Perhaps the article has expired

<A921.1168277462.10038@discuss.softvelocity.com> (41090)

Click here to remove all expired articles

I am now trying to get back to where I was with TBird.  :(

 

Posted on Wednesday, February 28, 2007 by Dave Harms

I've had my share of troubles with TB as well which is one of the reasons I like Hamster - if TB blows up I can get the Hamster cache quite quickly.

Evidently you changed your existing account to use TB instead of setting up a new one. I can't think why that should present a particular problem since the message numbers are set by the SV server. But have you tried setting up a new newsgroup account for Hamster?

Dave

 

Posted on Wednesday, February 28, 2007 by Phillip Carroll

I have set up a new account with Hamster as the server, and subscribed to just the new private (Alpha) group.  That seems to work fine.  

However, flipping thru the Hamster group folders, I am finding a lot of inconsistency among the .INI files.  It seems that when it has run into timeouts, it made a mess of the ini files.  For instance, comp.lang.clarion had the following .ini file section:

[Ranges]
Local.Max=648
news.softvelocity.com.Min=1
news.softvelocity.com.Max=130977
news.softvelocity.com.Low=1

As a result of this local max limit, when I added c.l.c to the new TB account, all I got was 648 messages, which were basically a random sampling of the last 4 years of c.l.c.  I reset these settings using the News Configuration|Newsgroups|Settings dialog.

The logs indicated I had a lot of timeouts.  I have increased the timeout value to 5 minutes for data loading, and decreased the wait time in the script loop to 1.5 minutes.  I think what was happening with the timeouts was that Hamster was telling the Clarion server it wanted 130k messages for c.l.c, and this caused the Clarion server to go think about this for awhile.

Note: My global settings were set to download all messages in each group. I have now gone back to News Configuration|Settings, Set the pull limits to 2000 newest for an empty group and 500 newest otherwise, and enabled splitting with the default values of 300 as the split threshold, and 250 entries in each part.

Hamster is now happily downloading all of c.l.c in nice little batches. (No timeouts or errors of any kind) So, these settings seem to be working.  

BTW: I have Hamster on a network machine.  TBird Tip: If you ever want to have more than one account go to the same server (I do this all the time), edit the Windows Hosts file. Add entries with different names, aliased to the same host.  Use the alias names in TBird.   

 

Posted on Wednesday, February 28, 2007 by Dave Harms

Phil, thanks for the TB tips and the timeout info. I also found it took quite a while to get everything downloaded and at first I seemed to get a random sampling, but after a while Hamster picked up all the messages.

 

Posted on Wednesday, February 28, 2007 by Phillip Carroll

Changing to xxx newest turned out to be a bad idea.  It then ignores all older messages. I changed back to pull all, but left the splitting feature turned on, which I think is key.  I had to go in and manually change the ini files to get it to go pick up the older messages.

Also, I changed the settings in Configuration|Local Servers|NNTP|Storage Settings to all zeroes (keep forever).  Otherwise, it would start purging articles after 7 days.  

I want a permanent databse of these articles for searchnig purposes.  Which reminds me:  You mentioned that Hamster supports NNTP searching.  How do I use that with TBird?  The only way I have been able to do TBird searches of newsgroups in the past has been to enable offline for a group, then go offline to search the downloaded messages.

 

Posted on Thursday, March 01, 2007 by Dave Harms

Right-click on a newsgroup and choose Search. Unfortunately TB only searches this way on Subject and From. Dunno if that's an NNTP limitation or a TB limitation.

Dave

 

Posted on Thursday, March 01, 2007 by Charlie Anderson

Ever since we switched to GroupWise as our email and colaboration software, I have been unable to use the SV newgroups effectively.

I followed these excellent instructions and GroupWise is suddenly a useful news reader; and my postings will comply with our archiving requirements.


 

Posted on Thursday, March 01, 2007 by Dave Harms

Charlie,

Cool, I'm glad it's been helpful!

Dave

 

Posted on Thursday, March 01, 2007 by Phillip Carroll

I solved the problem of NNTP searching on TBird.  (I am using version 1.5.0.9)  This is really kewl.

Whenever a group is selected, a search entry field appears in a header line just above the pane layout.  (I use classic view) I don't remember this search field in the previous version.  Whatever.  If you click the magnifier (search) icon, a drop down list is presented that lets you choose what to search. "Entire message" is in the list. Choose "Entire message".

Select a group in an account attached to Hamster (where TB has downloaded only headers). Hint:  It may a good idea to limit your view to a subset of the articles.  I initiated a full text search of clc in its entirety (about 130k articles) and TB locked up.  Possibly, the fact I have Hamster on a network machine may exacerbate the problem, but evidently TB doesn't like hefty searches.   On a smaller group (the new private group) I got instant results.  

Enter a search value in the entry field.  The entry field is an incremental locator.  As soon as you enter something, the NNTP search will start, the hourglass icon will appear, and the list of headers will shrink to just those that contain the search argument.  Select any of these headers, and the article will appear with all instances of the search text highlighted.  Love it!

Note: Searches of groups in an account served directly by Clarion always come up empty.  I guess because it does not support NNTP searching.

 

Posted on Thursday, March 01, 2007 by Dave Harms

Sweet! Thanks, Phil!

 

Posted on Friday, March 02, 2007 by George Riddell

How you get ThunderBird to work with Hamster.

The article was at the level that I expected
to be connected to ThunderBird instead of NOTHINGH.

How do you connect?  I've wasted too much time on
this mindless matter.
 

 

Posted on Friday, March 02, 2007 by Phillip Carroll

I misspoke in my previous note.  It turns out that in order to make use of the group search feature in TBird, the group must be selected for offline use, and (presumably) the messages must have been downloaded. (That is:  File|Offline|Download/Sync Now)  The search can be conducted with TBird in online mode, but it is not an NNTP search, it is a local search.  I can't seem to find a TBird feature for auto-syncing at intervals. (Like the hamster script loop)

At this point, assuming I download/sync full articles into TBird for the search capability, it looks like the only potential value adds provided by Hamster are:
1) You can easily program filters into Hamster that are not possible with TBird  (possibly useful for advanced twit filters)
2) It can presumably reduce WAN traffic when you have multiple newsgroup clients on a LAN. (They can all use a LAN-based Hamster server instead)

Since neither of these features is currently important to me, I will probably ditch Hamster.  I suppose if I use a news client that supports nntp search, there may some value, depending on whether it has at least the features of TBird.  But then again. I don't really know how much functinality there is in nntp search.

One thing I urgently need to do is move TBird's data folders off my C drive.  It is really creaking...

To add a comment to this article you must log in.

 
 

Search

 

Advanced Search
Topical Index

Related Articles

Subscribe to
ClarionMag

One year: $169

(includes all back issues since '99)

Renewals from $119

Two years: $269

Renewals from $219

More Info

Subscribe Now!

ClarionMag Blog

RSS Feeds

Updates via Email

Enter your Email


Powered by FeedBlitz

Quick Links