Starting over (again)

Lovely new MacBook Air — light as a feather pillow.

Power up.

Switch on.

My Air arrived with zero charge. I had a nervous few minutes with no response from the power button. I had time to read the FAQ, try again, and panic briefly before it eventually had enough charge to start up.

Setup admin account

Setup wireless

Software update

This may take some time (for me this was an 870MB download). While waiting you can Install Firefox

Software update again

Repeat until there are no further updates

Migrate user data from your TimeCapsule backup

Do this over ethernet, unless you want a long wait (I had 40GB to migrate. I get 2-3GB/h over a wire connected to my remote Express — 1.5GB/h with two wireless hops, and 7-8GB/h when wired directly to the Time Capsule.)

Setup user accounts

Include a working account for yourself

Configure Time Machine Backup for your new machine.
Start the initial backup.

This will take even more time (my initial backup is requires transfer of 55GB of data).

You can carry on with other tasks meanwhile. You can interrupt the process ("Stop Backing Up" in the Time Machine Menu), and resume later. Again, do as much of this as possible over a wired connection, to speed things up.

Use Calaboration to sync your Google calendars with iCal

Install Fink and Fink Commander

This allows you to install and manage various Unix utilities. I start with emacs-carbon.

Install MacPorts and Porticus

This also allows you to install and manage various Unix utilities and for many has more up-to-date versions. It also has a port of polyml. I start with polyml, tetex, bibtex2html and hevea. Porticus doesn't have carbon-emacs.

Install Kerberos Extras

Our "staffmail" imap server supports Kerberos authentication - just use the realm EASE.ED.AC.UK and your EASE user name and password. Unfortunately Apple haven't yet implemented this fundtionality on the iPhone — so syncing broke my email on the iPhone.

Change back to password authentication; sync again, so mail on the iPhone is back to normal; turn off sync; turn on Kerberos authentication on the Mac.

Install Developer Tools

Apart from anything else, this is probably the easiest way to get CVS installed.

Download iPhone SDK

Useful for the iPhone emulator which lets you see how your web pages will look on the iPhone. Maybe someday I'll write some code too!

Turn on the Safari Develop menu

To display the Develop menu in Safari 3.1 or higher, select the checkbox labeled "Show Develop menu in menu bar" in Safari's Advanced Preferences panel.

Channel 13: Leopard Wireless Card Locale

Channel 13 is legal in the UK and many other locales, but not in the USofA.

Since most default setups use channel 6 or channel 1, those of us looking for uncrowded channels often find 13 is uncluttered.

When I did a recently clean install of Leopard, I found that my channel 13 network disappeared — invisible!

Update (2009-03-09) It seems that the 10.5.6 update (I installed it on 2008-12-27) has changed something so this tip no longer works!

Although my MacBook Time Zone was set to Edinburgh - Scotland (you can set this in System Preferences, under Date&Time), I found that my Wireless Card Locale was set to US (which disables channel 13).

You can check your Wireless Card Locale by going to "About This Mac" > "More Info ..." > "Network; Airport Card".

The solution was to set the Time Zone to Tokyo then restart. Wireless Card Locale is then "Worldwide" and channel 13 is enabled.

Setting the Time Zone back to "Edinburgh - Scotland" and restarting again gives me "Worldwide" — but setting Time Zone to "Los Angeles - USA", and restarting, sets it back to "US".

iPhone Google Voice Search Arrives!

Google Mobile App

Out with the old; in with the new.

It doesn't appear as an update, in the iTunes store, the post date and version number appear unchanged (see the image) — ignore that! Just delete the old one then get the Google Mobile App again from the AppStore. Apple will tell you the update is free since you already bought this App. The packaging is still old, but Now the update appears and the packaging is updated too the app inside is new (version 0.3.142.951) and it does do voice search.

Time Machine hangs? Spotlight responsible?

Time Machine and Spotlight run slowly over wireless...

So slowly, that it appears that the system has hung.

Time Machine is great — but ...

Making the first backup of a 60GB of data takes a very long time over wireless.

Making a large incremental backup, after being away for a week, or more takes a very long time over wireless. Even over 802.11n I find Time Machine, backing up over my WDS, manages about 1MB/sec. Say 1GB takes 16 minutes, then 60GB takes 16 hours!

If Spotlight is indexing the backup while the backup is changing, things go even slower.

Solution: For the first backup, or for an incremental backup after you've been on the road, first turn of Spotlight indexing for the backup. Then connect your Mac by ethernet cable directly to the LAN port on the Time Capsule, and leave it to chunter away overnight. Finally, turn indexing on to let Spotlight digest the backup.

To see what Time machine and Spotlight are up to, use the console to inspect the logs. Set the filter so you see messages from backupd. You should see a sequence of messages appear slowly (but no longer very slowly), like this:

Starting standard backup 
Network volume mounted at: /Volumes/Data 
Disk image /Volumes/Data/myMacBook0016cb896cb9.sparsebundle mounted at: /Volumes/Backup of myMacBook 
Backing up to: /Volumes/Backup of myMacBook/Backups.backupdb 
No pre-backup thinning needed: 2.21 GB requested (including padding), 801.41 GB available 
Copied 22 files (24.4 MB) from volume Macintosh HD. 
Starting post-backup thinning 
No post-back up thinning needed: no expired backups exist 
Backup completed successfully.

Each line starts with date and time and the label /System/Library/CoreServices/backupd[22205]

Note:

Spotlight should be allowed to index your backup — so that you can find valuable nuggets of information lost in the past. Spotlight is also very slow if it has to index 60GB over wireless — if you do this, your log may also include lines like:

Waiting for Spotlight to finish indexing /Volumes/Backup of myMacBook/Backups.backupdb 

The solution is the same — when Spotlight has lots of changed stuff to index in the backup, let it work over ethernet. If you unplug and go wireless immediately your big backup is done, Spotlight will spend a long time catching up — and it won't let the next hourly backup begin until it has caught up.

Spotlight crashes

In addition, spotlight crashes: mdworker does the Spotlight indexing — you may find messages like this:

Formulating crash report for process mdworker[22921] 
(0x10c720.mdworker[22921]) Exited abnormally: Bus error 

When this happens, it slows things down even more. It's a bug — every crash is a bug. Some discussions suggest that it may by triggered when Spotlight attempts to index ill-formatted emails.

You can tell Spotlight not to index emails — and it may have some effect. Waiting patiently also seems to work — and I need to be able to search for mail by content, so I have to let Spotlight index my mail.

iPhone ... cannot be synced (error 13014)

The iPhone ... cannot be synced.
An unknown error occurred (13014).

This occurred after the iTunes 8.01 (11) update. Restarting the Mac seems to cure it.

iPhone Screen Capture

To screen grab: While holding the Home button, click the on/off/lock button.

You will find an image file among your photos. Download it to iPhoto. Export as PNG, TIFF or JPEG.

VPN on iPhone L2TP/IPsec

Settings > VPN > Add VPN configuration...

To connect to vpn2.net.ed.ac.uk ...

Account

<your UUN>

Password

<your EASE password>

Secret

<find it here>
No cheating in the library!

Save

ENTITY &mdash;(X)HTML entities in XML for XSLT

You can use character entities in your XML provided you declare them.

Add a DOCTYPE to your XML — all you need is the name of your root tag (my document in the example below is a recipe) and a reference to the special character sets you want.

for example

XML-compatible ISO Special Character Entity Set for XHTML

— including lt gt mdash quot euro

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<!DOCTYPE recipe [
       <!ENTITY % xhtml-special
           PUBLIC "-//W3C//ENTITIES Special for XHTML//EN"
       "http://www.w3.org/MarkUp/DTD/xhtml-special.ent" >
       %xhtml-special;
]>
<recipe>
  ...
</recipe>

other standard character sets

You can add more declarations to the DOCTYPE mdash just add them between the square brackets [...]

XML-compatible ISO Latin 1 Character Entity Set for XHTML

— fractions, accented characters, pound, yen, cent, copy

       <!ENTITY % xhtml-lat1
           PUBLIC "-//W3C//ENTITIES Latin 1 for XHTML//EN"
       "http://www.w3.org/MarkUp/DTD/xhtml-lat1.ent" >
       %xhtml-lat1;

ISO Math, Greek and Symbolic Character Entity Set for XHTML

       <!ENTITY % xhtml-symbol
           PUBLIC "-//W3C//ENTITIES Symbols for XHTML//EN"
       "http://www.w3.org/MarkUp/DTD/xhtml-symbol.ent" >
       %xhtml-symbol;

Printing in the Forum

Updated for Snow Leopard 2009-10-21.

The print-server address is printif.inf.ed.ac.uk

You can look up the available printers with the DICE command printers
[tammy]mfourman: printers
if213m0 HP 4100DTN - IF213
if237c0 Canon iRC3080i multi-function device in IF237
if313m0 HP8150DN - IF313
if336c0 Canon iRC3080i multi-function device in IF336
if413m0 HP8150DN - IF413
if435c0 Canon iRC3080i multi-function device in IF435
if536c0 Canon iRC3580i multi-function device in IF536


You should install and select the Canon iR C3080/3480/3580 PPD Canon UFR_II_V200_MacOSX_us_EN.dmg File for Mac OS X for the multi-function (scan-print-copy) Canon devices.

You probably already have the HP drivers. If the configuration is not attached to the printer, you can print it using the printer's menu buttons.

Safari 4 beta scores 100/100 on acid3

Update 2009-02-24 This is now a public beta from Apple.

If you are registered as an ADC member (go to http://developer.apple.com/ then to the ADC member site link to register for free), you can now download the Safari 4 Developer Preview.

This this is very fast and very compliant: 100/100 on the Acid3 test. (Firefox 3.0.5 scores 71/100.)

First Direct online banking now works fine (and fast).

Starting over: MacPorts

As a long-term Fink user, I've found it hard to summon up the energy to move to Macports (previously known as Darwin Ports), until now. A dead disk provided the necessary impetus to overcome this energy barrier.

MacPorts—so I am told by friends I trust—is closer to the Mac zeitgeist, and has more packages available in fresher versions.

My first experience was not good. I tried the standard install from .dmg with two fresh, fully updated installs of Leopard + x11 + XCode Tools—one on my ageing MBP; one on a sparking new MacBook Air. Neither succeeded in creating the code>.profile that is supposed to adjust the PATH environment variable.

You should start with a standard install as it does almost everything—and may even do it all, for some it succeeds.

In the environment of a shell accessing packages installed by MacPorts, PATH should include /opt/local/bin:/opt/local/sbin and MANPATH should include /opt/local/share/man. The standard install is meant to create a .profile to achieve this. It didn't.

Googling macports leopard profile led to various suggestions (as usual, others have encountered this problem before me). One of these works—others don't.

Do not add the new paths to /etc/paths and /etc/manpaths.

Do not add new files named macports or MacPorts, containing the new paths, to /etc/paths.d/ and /etc/manpaths.d/.

Do create a file ~/.profile containing the following code:

export PATH=/opt/local/bin:/opt/local/sbin:$PATH
export MANPATH=/opt/local/share/man:$MANPATH

Then you can type sudo port -d update in a fresh terminal window, to update your MacPorts installation, and sudo port install emacs-app, for example, to install a Cocoa version of emacs.

You'll find the Emacs installed as a regular application in /Applications/MacPorts/.

Note: If you already have a .bash_profile or .bash_login, you can and should append the commands above to that file, and optionally rename rename that file as .profile, instead of creating a new .profile.

Explanation: (for the full story try man bash)

When bash is invoked as an interactive login shell, or as a non-inter-active shell with the --login option, it first reads and executes commands from the file /etc/profile, if that file exists. After reading that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile, in that order, and reads and executes commands from the first one that exists and is readable.

Fixing mail.app’s IMAP date problem

Using imapsync to migrate mail from one IMAP server to another, I hit a problem. Google led me to the following description of the same issue:

The IMAP Date problem is the result of how mail.app figures out the Date Received time for an email. Rather than using the Date: header in the email it uses the time the file was written to the file system. This becomes a problem when files are copied to a new location on the server and the creation time of the file is changed.

Not mail.app's problem!

The solution is to use imapsync with the flag --syncinternaldates

I am now using

/usr/bin/imapsync --syncinternaldates \
     --host1 mail.inf.ed.ac.uk --user1 mfourman --ssl1 \
     --host2 imap.staffmail.ed.ac.uk --user2 mfourman -ssl2 \
     --noauthmd5

to finish the transfer of about 4GB of mail history.

To start, try it out on a small folder --folder Test

First test with --dry :

/usr/bin/imapsync --syncinternaldates  --folder Test \
     --host1 mail.inf.ed.ac.uk --user1 mfourman --ssl1 \
     --host2 imap.staffmail.ed.ac.uk --user2 mfourman -ssl2 \
     --noauthmd5 --delete --dry

If everything looks OK then run

/usr/bin/imapsync --syncinternaldates  --folder Test \
     --host1 mail.inf.ed.ac.uk --user1 mfourman --ssl1 \
     --host2 imap.staffmail.ed.ac.uk --user2 mfourman -ssl2 \
     --noauthmd5 --delete

Expunge from the source by hand (in Mail.app Mailbox > Erase Deleted Messages).

Using the --expunge flag upsets other users of the IMAP server:

Currently it is processing folder which is 100MB in size, it is deleting (and then expunging) one message at a time. The way that is implemented on the server, is that the message is deleted, and then the whole file is written out again with that one message removed. This is leading to a lot of disk IO.

That's the short story. The actual story was more involved and also included using my gmail IMAP account as a temporary buffer to recover from some false starts. (Free 6.5GB quota - thank you Google!)

Opaque Leopard Menu Bar

Many find the translucent menu bar in Leopard disconcerting. There are several tips posted elsewhere suggesting various hacks involving obscure environment variables to turn off this feature.

Apple did it for you! (since 10.5.2)

Go to System Preferences > Desktop and Screen Saver. Uncheck the Translucent Menu Bar checkbox. You're done!

emacs: Fatal malloc_jumpstart() error

Emacs fails with the error Fatal malloc_jumpstart() error

Black Magic

In a Terminal window type

$ sudo mv /usr/bin/emacs-i386 /usr/bin/emacs-i386.backup
$ sudo /usr/libexec/dumpemacs -d
$ emacs --version
GNU Emacs 22.1.1
...

(Here, $ is the shell prompt.)

Once youve checked all is OK you can remove the backup

$ sudo rm /usr/bin/emacs-i386.backup

Time Capsule

Setup

Time Capsule is an airport extreme base station (802.11 a, b, g, n) with built-in hard drive for backup.

You set up the base station as usual, using AirPort Utility.

You can use your old base station to create a WDS network and extend your wifi coverage to attic or garden – or share with a neighbor. In any case, APPL say, you should make the Time Capsule your main base station – the one directly connected to your ISP.

If you just have 802.11n-capable access points you can let your network autoconfigure. Choose a common Network Name, Wireless Security and Password. Under Airport > Wireless set the main base station to Create a wireless network; set the other base stations to Extend a wireless network.

A domestic ISP typically gives you internet access via a single IP address. Depending on your ADSL or cable hardware you have various options for setting up your system to distribute local IP addresses to clients on your local network.

My ADSL modem is a speedtouch router. It provides a DHPC server with optional MAC authentication. WDS setup is simple: each base station is configured to get its own IP address as a DHCP client, and to act as a bridge passing wifi client traffic to and from the ADSL modem.

If your ADSL or cable modem isn't set up as a DHCP server, you can get the Time Capsule (or any Airport base station) to do this. Set the main base station to "Share a public IP Address" (under Internet).

If your modem provides a range of addresses that clients can select manually, you can set the main base station to distribute these. Select "Distribute a range of IP addresses."

Backup

Time Machine sometimes gets confused and can't mount the backup disk. Apple say the volume name should be shorter than 27 characters. Elsewhere they say it should be purely alphanumeric [0-9,a-z,A-Z]*

Extreme 7.3.1 firmware update

All went smoothly, but now my remote base stations (I have a WDS daisy chain 4 links long - see earlier post) don't show in Airport Utility. Still connected - I can ping them just fine, and the remote airTunes express shows up in iTunes.

Update this seems to have been a temporary problem. Everything is now visible again.

All my airports (7 at last count) are configured in bridge mode - my speedtouch adsl modem router provides a DHCP server. On the bright side, this update seems to have helped with one issue: the 802.11n extremes now seem to pick up an IP address from my router using DHCP (previously I found this didn't work and had set manual IP addresses as a work-around).

application/x-mplayer2 plugin

Problems with BBC radio-player?

You need the Flip4Mac internet plugin. Download it free from microsoft. Use the "customize" button when installing to select the browser plugin.

Uninstall SPSS

installer tells me that a there is still a previous version of SPSS 16 on my mac

SPSS is great, but ...

The Mac install is clunky. If you've aborted an install and want to start again, or you just want to get some space free, you have to remove the old installation.

You've removed all files and directories matching *spss* (and, for good measure, youve also tried *SPSS*), and you can't start again because the installer complains that you haven't removed a previous installation.

remove InstallShield (probably sitting in your home directory) and start again!

If you want to get rid of an older version, and you haven't yet started, use the uninstaller installed with SPSS,

SPSS know about the problem

They say:

Problem Subject: Trying to install SPSS 16 for the Mac yields an error message that there is already a previous flavor

Problem Description: I am trying to re-install SPSS 16 for the Macintosh and I am unable to do so because during the installation I am met with an error message telling me that there is already a previous flavor of SPSS on this machine and that it must be removed before a new install can take place. What is the reason for this message and how can I resolve this so that I can install SPSS 16 for the Macintosh?

Resolution Subject: Completely remove the previous installation of SPSS 16 from this machine

Resolution Description: The reason this is happening is that the previous installation of SPSS 16 for the Macintosh was not uninstalled correctly. If you merely trashed the application folder then SPSS 16 was not uninstalled. The uninstall function is found with the SPSS 16 applcation in the 'Applications' folder. In order to resolve this issue if SPSS 16 is not removed through the uninstall process please go into the 'Users' folder on the main operating systems hard drive. In here you should find a house with your login name on it and you should go into this item. In this house (folder), please trash the following:

~/initspssmac

~/InstallShield (directory, remove the whole thing)

~/Library/Preferences/com.spss.spss for mac.plist

Now empty your trash. Now you will be able to re-install SPSS 16 for the Macintosh.

free iPhone SDK

Apple say:

``Start your development today with the free iPhone SDK ...''

... not quite: this is a 2.1GB download, and it looks as though AAPL wasn't ready for the demand. Currently at 7% after seven hours, and just picking up speed (presumably as the US goes to sleep): occasional bursts of action, but still very slow—10 hours to go...

Wireless Woes

Strange behaviour: Everything was fine, working with my MBP on a multi-AP WDS network of new Airport Extreme base stations; move to another floor, just next to an AP, can't see the signal, or rather I see it fleetingly, get asked for password, then it disappears. My first conclusion is that the AP must be caput, but back upstairs, where I started, I now see no signal.

Another laptop can see the signal fine. I try manually changing network setup, restart, shutdown, even taking out the battery then restart. Still no wifi connection. I conclude that the wifi card in my laptop must be damaged, or at least have a dodgy aerial connection.

Back at home I still have the same problem—and neither Airport Utility, nor iStumbler can see the signal either. I think I'll try looking at the wifi card.

To check on how to do that I need to get online. So I find an ethernet cable and connect a LAN port on one of my base stations. I'm now online, as expected, but I didn't expect to see Airport Utility and iStumbler burst into life just because I made a wired connection. Everything looks normal, and Network preferences now says I have a wireless connection.

I unplug the ethernet cable, and everything stays fine; back to normal. I don't pretend to understand why.

update

It happened again!

Same symptoms - persisting across two different WDS networks.

Again, recovery coincided with my decision to give up hope and use a wired connection to my base station.

Crazy!

AJAX demystified: import xml

Here's the barebones of a javascript function that will request xml from a url configured to return xml (text/xhtml+xml or application/xml for example) and then asynchronously add a selected fragment of the returned xml as a child of some node on your page; this target node is selected by id.

function getXML(url, toGet, id){ 
  var request = new window.XMLHttpRequest();
   // for IE use ActiveXObject instead

  function load(){
      var content, node;
      if(request.readyState == 4 &&
         request.status == 200){
         content = document.adoptNode(
           request.responseXML.getElementById(toGet));
         node = document.getElementById(id);
         node.appendChild(content);
    } // should handle null response or failure
  }

  if(request){
      request.open("POST", url, true);
      request.setRequestHeader("Accept","application/xml");
      request.onreadystatechange=load;
      request.send(null);
  }// else ...
}

This will work on a gecko browser, if you have everything set up properly. You'll have to expand it to make it robust, as per the comments (and more).

Example xhtml to call this:

<div id="putInfoHere" />
<span class="button" 
  onmouseup='javascript:getXML(
    "http://my.site.net/contact.xhtml",
    "address",
    "putInfoHere");'>Get Info</span>

The file contact.xhtml should have an element such as <div id="address">...</div> with the matching id. This will be harvested. The clever part is that the harvested code still lives in the correct namespace. So imported markup is correctly interpreted as xhtml.

Make sure files are served with an appropriate mime-type. With apache you can do this using .htaccess :

AddType application/xhtml+xml .xhtml

A future post will have more, on how to send text or xml data to a script using httpRequest with POST instead of GET.

AJAX demystified: import text; async cgi

Here's the barebones of a javascript function that will request text from a url configured to return text (text/plain or text/plain for example) and then asynchronously put the returned text into a textarea element (or any other element with a value field) on your page; this target element is selected by id.

function getText(url, id){ 
  var request = new window.XMLHttpRequest();
  // for IE use ActiveXObject instead

  function load(){
    if(request.readyState == 4 &&
         request.status == 200){
      document.getElementById(id).value
        = request.responseText;
  // should handle null response or failure
  }}

  if(request){
      request.open("GET", url, true);
      request.setRequestHeader("Accept","text/*");
      request.onreadystatechange=load;
      request.send(null);
  } // else ...
}

This will work on a gecko browser, if you have everything set up properly. You'll have to expand it to make it robust, as per the comments (and more).

Example html and javascript to call this (note how the call uses the id of the textarea):

<textarea id="putInfoHere" />
<span class="button" 
  onmouseup='javascript:getText(
    "http://my.site.net/info.txt",
    "putInfoHere");'>Get Info</span>

Make sure text files are served with an appropriate mime-type. With apache you can do this using .htaccess :

AddType text/plain .txt

You can replace info.txt with a cgi script that produces a header and info on stdout. For example (with otherwise pointless sleep to demonstrate asynchrony), put the following mydate.cgi in a cgi-enabled directory:

#!/bin/sh
echo "Content-type: text/plain\n"
date
sleep 2
date

Change the URL in your html accordingly, and make sure your script is executable chmod +x mydate.cgi

A future post will have more, on how to receive xml instead of text and how to send text or xml data to a script using httpRequest with POST instead of GET.

Mac OS X 10.5: Web Sharing - "Forbidden 403"

In what follows, substitute your short username for myshortname

Create the file
/etc/apache2/users/myshortname.conf
containing the following text:

<Directory "/Users/myshortname/Sites/">
Options Indexes MultiViews
AllowOverride None
Order allow,deny
Allow from all
</Directory>

Set the group and owner as follows:
sudo chown root:wheel /etc/apache2/users/myshortname.conf

Restart apache by switching Web Sharing off then on in System Preferences.