TweetFollow Us on Twitter

Still More Perl

Volume Number: 19 (2003)
Issue Number: 1
Column Tag: Section 7

Still More Perl

Munging Mail and Media...

by Rich Morin

Perl's "whipitupitude" is legendary. This column looks at a couple of small scripts I've recently been "whipping up", showing how Perl can work in and around more formal OSX tools. One script, fmmf, Finds Monster Mail Files; I use it to keep track of mailing list (and other) mail files which may be getting out of hand. The other script, cfwc.d, is a daemon (background process) which helps me operate an experimental webcam.

Finding MOnster Mail Files

I'm on quite a few mailing lists and I don't always get to the associated mailboxes regularly to keep them under control. I'm also trying to track the efficacy of my spam filtering system (based on SpamAssassin and Eudora), which drops suspected spam into one of several mailboxes, depending on its numeric spam rating, etc. I have written a short script which helps me keep on top of these issues.

The mainline code, below, is quite simple. Using finddepth, from the File::Find module (available on the CPAN; cpan.perl.org), it performs a depth-first examination of my email folder. The callback function, wanted, is invoked for each node (e.g., file, directory) in the tree. Using the lists produced by this traversal, the remaining code prints out the results for spam and miscellaneous email, sorting each list in a case-insensitive manner.

#!/usr/bin/env perl
#
# fmmf - find monster mail files
#
# Written by Rich Morin, CFCL, 2002.11
use File::Find;
$monster = 2000000;
{
  $eu = '/Users/rdm/Mail/Eudora Folder';
  finddepth(\&wanted, "$eu/Mail Folder");
  for $line (sort {lc($a) cmp lc($b)} (@spam)) {
    print $line;
  }
  print "\n";
  for $line (sort {lc($a) cmp lc($b)} (@misc)) {
    print $line;
  }
}

The tricky parts of this script, such as they are, lie in the "wanted" callback function. As it traverses the tree, finddepth changes the "current directory" and sets $_ to the relative name of the node. This makes it easy to skip over items that aren't files and Eudora's "table of contents" (*.toc) files.

sub wanted {
  return unless (-f $_);
  return if ($_ =~ m|\.toc$|);

For the next part, however, we need the "full path name" of the node. Getting this from a handy helper method, we can strip off the first part of the path and test the remainder in assorted ways. Perl's regular expressions are very useful for this sort of name handling.

  $path = $File::Find::name;
  $path =~ s|^.*/Eudora Folder/Mail Folder/||;
  return if ($path =~ m|_Inactive/ Save/|);

After picking up the size of the file (in bytes), the script opens each mailbox in the "spam" area and counts the number of "From: headers (i.e., messages). Eudora uses carriage returns (rather than the conventional BSD newlines) for line termination, but setting Perl's $/ (input record separator) variable handles that quite easily. The strings containing the formatted output are pushed into a list, for use by the mainline code.

  $size = -s $_;
  if ($path =~ m|!Spam|) {
    open(MBOX, $_) or die "can't open mailbox($_)";
    $/ = "\r";
    $fcnt = 0;
    while (defined($line = <MBOX>)) {
      $fcnt++ if ($line =~ m|^From:|) ;
    } 
    close(MBOX);
    push(@spam, sprintf("%-35s  %9d  %4d\n",
      $path, $size, $fcnt));
    return;
  }

The code for miscellaneous mailboxes is comparatively simple. After ensuring that the mailbox is large enough to qualify as a "monster", it formats and saves the output lines. Perl's "x" operator comes in handy for creating a "quick and dirty" histogram.

  return if ($size < $monster);
  $isiz = int($size/$monster);
  push(@misc, sprintf("%-35s  %9d  %s\n",
    $path, $size, '*' x $isiz));
}

This sort of "personalized" script is quite common in BSD circles. Clearly, it isn't suitable for use by others, as is, but it is short and simple enough that it can easily be customized to meet the needs of different users. Here is some sample output, from my own system:

!Spam/?? Junk (Eudora)                9041     5
!Spam/?? Junk (SA 1)                 39192     6
!Spam/?? Junk (SA 2)                 11467     2
!Spam/?? Junk (SA 3)                420538    60
_Lists/DocBook                     3231686  *
_Lists/FreeBSD/FreeBSD-Ports       6431902  ***
_Lists/FreeBSD/FreeBSD-Questions   2666962  *
...

A WebCam Daemon

I recently started playing with an iBOT, a FireWire-based camera made by Orange Micro

(www.orangemicro.com). My initial goal was to create a simple "security camera" app that would display a set of recent images on a web page.

After downloading the OSX driver for the iBOT, I started looking around for image capture software. One package, EvoCam (www.evological.com), captures images, based on elapsed time and/or software-based motion detection. It can also upload the image files (via FTP) to a web server and/or save numbered copies on the local disk.

Unfortunately, this wasn't exactly what I wanted. The FTP upload feature simply refreshed the same file; turning this into a time history would be tricky. The numbered image files would do, however, if I could get them over to the web server. All told, it was a good start on what I wanted. All I needed to do was create a little plumbing...

The first part of the plumbing had to do with getting the files from my desktop Mac onto the (FreeBSD-based) local web server. FreeBSD provides NFS, but getting OSX to mount the provided volumes can be quite a trial. Fortunately, Marcel Bresink's NFS Manager (www.bresink.de/osx/NFSManager.html) eases the pain considerably.

Once I got the files sifting into a directory on the web server, I merely had to rename them (for convenience) and build up a web page to display a selected subset. The following script, while still a "work in progress", accomplishes these tasks quite handily.

#!/usr/bin/env perl
#
# cfwc.d - Canta Forda WebCam Daemon
#
# Written by Rich Morin, CFCL, 2002.11
$imgs = '/.../iBOT';   # adjust to taste...
$html = '/.../cfwc';   # adjust to taste...
{
  for (;;) {

As mentioned above, EvoCam generates a unique name (e.g., 123456789.jpg) for each image file. In writing these to the NFS-mounted FreeBSD machine, OSX also generates a companion file (e.g., ._123456789.jpg) for the resource fork. The code below creates a new name for the image file, based on the file's modification time, and discards the companion file.

    # Clean out incoming directory.
    opendir(IN, "$imgs/incoming")
      or die "can't open $imgs/incoming";
    @in = grep(!/^\./, readdir(IN));
    chomp(@in);
    closedir(IN);
    for $in (sort(@in)) {
      @stat = stat("$imgs/incoming/$in");
      $mtime = $stat[9];
      ($sec, $min, $hour, $mday, $mon, $year,
       $wday, $yday, $isdst) = localtime($mtime);
      $out = sprintf("%d.%02d%02d.%02d%02d%02d.jpg",
        $year+1900, $mon+1, $mday, $hour, $min, $sec);
      rename("$imgs/incoming/$in",
             "$imgs/i.queue/$out");
      unlink("$imgs/incoming/._$in"); 
    }

Perl's approach to reading directories is rather messy, but it isn't all that difficult. The code below gets a list of filenames, discarding any that don't match the desired format, and sorts them. Because the names were crafted with this in mind, the list is now in chronological order.

    # Get list of images to display.
    opendir(IN, "$imgs/i.queue")
      or die "can't open $imgs/i.queue";
    @in  = sort(grep(/^\d{4}\.\d{4}\.\d{6}\.jpg$/,
                     readdir(IN)));
    chomp(@in);
    closedir(IN);

Using Perl's "slice" syntax, we grab the last (i.e., most recent) nine file names.

    @show = @in[-9 .. -1];

Now we start generating a web page. The META tag tells the user's browser to refresh the page every 15 seconds. I am rather compulsive about formatting the HTML; the web browser doesn't care, but it sure makes debugging less painful for humans!

    # Make up a new web page.
    open(OUT, ">$html/index.temp")
      or die "can't open index.temp";
    print OUT <<EOT;
<HTML>
  <HEAD>
    <META HTTP-EQUIV="Refresh" content="15">
    <TITLE>Canta Forda WebCam</TITLE>
  </HEAD>
  <BODY>
    <TABLE>
EOT

The code below generates a 3x3 table of images, each followed by a centered label. I could have used the file names (e.g., 2002.1129.2039.jpg) as labels, but that would have been a bit ugly. Why not parse the names and reformat the values into a more readable format?

Note the multi-line regular expression that is used to break up the file name. When REs get long and complex, breaking them up in this manner can make them much easier to follow.

    $cnt = 0;
    for ($i=0; $i<9; $i+=3) {
      print OUT "      <TR>\n";
      for ($j=0; $j<3; $j++) {
        print OUT "        <TD>\n";
        $k = $i + $j;
        $tmp1 = $show[$k];
        $tmp1 =~
          m|^(\d{4})\.            # (YYYY).
             (\d\d)(\d\d)\.       # (MM)(DD).
             (\d\d)(\d\d)(\d\d)\. # (HH)(MM)(SS).
             jpg$|x;              # jpg
        $tmp2 = sprintf("%s/%s/%s at %s:%s:%s",
                        $1, $2, $3, $4, $5, $6);
        print OUT "          <CENTER>\n";
        print OUT "            ",
                  "<IMG SRC=\"iq/$tmp1\"><BR>\n";
        print OUT "            $tmp2\n";
        print OUT "          </CENTER>\n";
        $cnt++;
        print OUT "        </TD>\n";
      }
      print OUT "      </TR>\n";
    }

Finally, we push out the last of the HTML, close the file and (Oh, yes!) move it into place for Apache to find. Then, after a second's repose, we go back up and do the whole exercise again.

    print OUT <<EOT;
    </TABLE>
  </BODY>
<HTML>
EOT
    close(OUT);
    rename("$html/index.temp",
           "$html/index.html");
    sleep(1);
  } 
}

Lessons Learned

As we all know, the Mac and BSD universes aren't a perfect fit. Perl is a very good "glue language", however, allowing us to deal smoothly with issues such as line termination, extra (e.g., resource fork) files, etc.

Similarly, there are a wealth of useful apps which can perform small tasks, fill in gaps between operating systems, and generally make our lives easier. If a $20 shareware package can save me hours of frustration, the purchase decision is a no-brainer.

Unfortunately, some issues are still difficult to resolve. For instance, although it's easy to scan a Eudora mail file for header lines, editing Eudora mailboxes would be far trickier. Aside from file locking problems, there is the small issue of the (binary, undocumented) format of the TOC files. In short, choose your challenges carefully...


Rich Morin has been using computers since 1970, Unix since 1983, and Mac-based Unix since 1986 (when he helped Apple create A/UX 1.0). When he isn't writing this column, Rich runs Prime Time Freeware (www.ptf.com), a publisher of books and CD-ROMs for the Free and Open Source software community. Feel free to write to Rich at rdm@ptf.com.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

BusyContacts 1.6.4 - Fast, efficient con...
BusyContacts is a contact manager for OS X that makes creating, finding, and managing contacts faster and more efficient. It brings to contact management the same power, flexibility, and sharing... Read more
Steam 4.0 - Multiplayer and communicatio...
Steam is a digital distribution, digital rights management, multiplayer and communications platform developed by Valve Corporation. It is used to distribute a large number of games and related media... Read more
OmniGraffle Pro 7.19.3 - Create diagrams...
OmniGraffle Pro helps you draw beautiful diagrams, family trees, flow charts, org charts, layouts, and (mathematically speaking) any other directed or non-directed graphs. We've had people use... Read more
OmniGraffle 7.19.3 - Create diagrams, fl...
OmniGraffle helps you draw beautiful diagrams, family trees, flow charts, org charts, layouts, and (mathematically speaking) any other directed or non-directed graphs. We've had people use Graffle to... Read more
Hopper Disassembler 5.3.3- - Binary disa...
Hopper Disassembler is a binary disassembler, decompiler, and debugger for 32- and 64-bit executables. It will let you disassemble any binary you want, and provide you all the information about its... Read more
calibre 5.35.0 - Complete e-book library...
Calibre is a complete e-book library manager. Organize your collection, convert your books to multiple formats, and sync with all of your devices. Let Calibre be your multi-tasking digital librarian... Read more
Sound Studio 4.10.0 - Robust audio recor...
Sound Studio lets you easily record and professionally edit audio on your Mac. Easily rip vinyls and digitize cassette tapes, or record lectures and voice memos. Prepare for live shows with live... Read more
Sparkle Pro 4.0 - Visual website creator...
Sparkle Pro will change your mind if you thought building websites wasn't for you. Sparkle is the intuitive site builder that lets you create sites for your online portfolio, team or band pages, or... Read more
Dropbox 140.4.1951 - Cloud backup and sy...
Dropbox for Mac is a file hosting service that provides cloud storage, file synchronization, personal cloud, and client software. It is a modern workspace that allows you to get to all of your files... Read more
FotoMagico 6.0.5 - Powerful slideshow cr...
FotoMagico lets you create professional slideshows from your photos and music with just a few, simple mouse clicks. It sports a very clean and intuitive yet powerful user interface. High image... Read more

Latest Forum Discussions

See All

Best iPhone Game Updates: ‘Garena Free F...
Hello everyone, and welcome to the week! It’s time once again for our look back at the noteworthy updates of the last seven days. I got busted last week for not including the obligatory free-to-play matching puzzle game update of the week, and my... | Read more »
‘Horizon Chase’ China Spirit DLC Release...
Following the release of the excellent reveal of the Horizon Chase Senna Forever expansion, the game will be getting a new DLC on mobile platforms today. Today, the Horizon Chase China Spirit DLC pack will release on iOS and Android bringing in 9... | Read more »
‘PUZZLED’ from SNK and Hamster Is Out No...
Following ZED BLADE ACA NeoGeo earlier this month, SNK has brought over another game in the ACA NeoGeo series to both iOS and Android in the form of PUZZLED. SNK and Hamster originally brought the series to mobile with Samurai Shodown IV, Alpha... | Read more »
A House Full of Covid – The TouchArcade...
It’s been a rough week as both of our young children tested positive for Covid, and since recording this early on Friday my wife has tested positive now too. Thankfully the kids seemed to recover fairly quickly and are mostly back to normal, and I... | Read more »
TouchArcade Game of the Week: ‘Krispee S...
Krispee Street is a new hidden object game from Frosty Pop that is based on their popular and almost painfully sweet webcomic Krispee. This is one of the latest titles to be added to the Netflix Games catalog, which means you’ll need to log into... | Read more »
SwitchArcade Round-Up: ‘Escape Lala’, ‘B...
Hello gentle readers, and welcome to the SwitchArcade Round-Up for January 21st, 2022. In today’s article, we’ve got a lot of new releases. A lot. There were eight on the schedule when I went to bed last night. There were twenty-four when I woke up... | Read more »
Beta Testers Needed for Huge Version 2.0...
Ya’ll remember Dungeon Raid, right? The phenomenal matching RPG hybrid that launched on mobile more than a decade ago, but was more or less abandoned by its developer only to die a slow death on the App Store before the 32-bit Appocalypse finally... | Read more »
‘Ark Legends’ Gives Players a Chance to...
It’s Airpods and Amazon gift cards galore as Melting Games opens pre-registration for Ark Legends. The upcoming mobile RPG is giving away tons of in-game goodies such as gold, energy, iron core, hero summon chest and rare iron core to players who... | Read more »
‘Nickelodeon Extreme Tennis’ Out Now on...
Nickelodeon Extreme Tennis () from Old Skull Games and Nickelodeon is this week’s new Apple Arcade release. Nickelodeon Extreme Tennis features characters from old and new Nickelodeon shows including SpongeBob, TMNT, and many more. The tennis game... | Read more »
SwitchArcade Round-Up: ‘RPGolf Legends’,...
Hello gentle readers, and welcome to the SwitchArcade Round-Up for January 20th, 2022. In today’s article, we’ve got a massive amount of new releases to check out. We’ve got summaries of all of them, from heaven to hell. We also have the lists of... | Read more »

Price Scanner via MacPrices.net

Verizon’s 2022 iPad promo: $100-$310 off any...
Verizon has cellular-capable iPads on sale for $100-$310 off MSRP when purchased with an Unlimited service plan. Sale price is applied to your account monthly over a 24 or 30 month period, depending... Read more
Sunday Sale: Apple AirPods are on sale for up...
Amazon has Apple AirPods on sale for $10-$100 off MSRP today, depending on the model. All are in stock today with free delivery: – AirPods Max headphones (Blue): $449 $100 off MSRP – AirPods Max... Read more
These Apple resellers are offering 13″ M1 Mac...
Apple resellers are offering discounts on 13″ MacBook Pros with M1 Apple Silicon processors ranging up to $150 off MSRP. Here’s where to get one today: (1): Apple’s 13″ MacBook Pros with M1 Apple... Read more
Amazon lowers prices on select 13″ M1 MacBook...
Amazon has select Apple 13″ M1 MacBook Airs on sale for $150 off MSRP this weekend, starting at only $849. Their prices are the lowest available for new MacBook Airs today. Stock may come and go, so... Read more
Apple has 13″ M1 MacBook Airs back in stock s...
Apple has restocked a full line of 13″ M1 MacBook Airs, Certified Refurbished, starting at only $849 and up to $190 off original MSRP. These are the cheapest M1-powered MacBooks for sale today at... Read more
In stock and on sale! 16″ 10-Core M1 Pro MacB...
Amazon has new 16″ 10-Core/512GB M1 Pro MacBook Pros in stock today and on sale for $50 off MSRP including free shipping. Their prices are the lowest available for new M1 Pro 16″ MacBook Pro from any... Read more
Deal Alert!: 14″ M1 Pro with 10-Core CPU in s...
Amazon has the new 14″ M1 Pro MacBook Pro with a 10-Core CPU and 16-Core GPU in stock today and on sale for $2299.99 including free shipping. Their price is $200 off Apple’s standard MSRP, and it’s... Read more
Apple has 24-inch M1 iMacs (8-Core CPU/8-Core...
Apple has restocked a wide array of 24-inch M1 iMacs with 8-Core CPUs and 8-Core GPUs in their Certified Refurbished store. Models are available starting at only $1269 and range up to $260 off... Read more
Select 24″ M1 iMacs are on sale for $100 off...
Sales of Apple’s new 24″ M1 iMacs have been rare since its introduction, perhaps due to global supply issues. However, B&H is offering a $100 discount on select 24″ iMacs, and they’re in stock... Read more
M1 Mac minis are back in stock today at Apple...
Apple has M1-powered Mac minis available in their Certified Refurbished section starting at only $589 and up to $140 off MSRP. Each mini comes with Apple’s one-year warranty, and shipping is free: –... Read more

Jobs Board

Registered Nurse (RN) Employee Health PSJH -...
…is calling for a Registered Nurse (RN) Employee Health PSJH to our location in Apple Valley, CA.** We are seeking a Registered Nurse (RN) Employee Health PSJH to be Read more
Systems Administrator - Pearson (United State...
…and troubleshoot Windows operating systems (workstation and server), laptop computers, Apple iPads, Chromebooks and printers** + **Administer and troubleshoot all Read more
IT Assistant Level 1- IT Desktop Support Anal...
…providing tier-1 or better IT help desk support in a large Windows and Apple environment * Experience using IT Service Desk Management Software * Knowledge of IT Read more
Human Resources Business Partner PSJH - Provi...
…**is calling a** **Human Resources Business Partner, PSJH** **to our location in Apple Valley, CA.** **Applicants that meet qualifications will receive a text with Read more
Manager Community Health Investment Programs...
…is calling a Manager Community Health Investment Programs PSJH to our location in Apple Valley, CA.** **Qualified candidates will be invited to do a self-paced video Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.