My Ancient Computer Project

Preamble: If you're really looking for advice on floppy drives: 8", 5¼" and 3½", try these links:
http://www.retrotechnology.com/herbs_stuff/s_drives_howto.html
http://www.classiccmp.org/dunfield/
http://www.deviceside.com/

The good folk at "Device Side Data" sell a USB adaptor for 5¼ inch floppy drives and a range of relevant cables, power-supplies and enclosures.
For about US$100, you can have a working  5¼in USB setup, but it's bring-your-own-drive.
Sourcing a 5¼ inch floppy drive may be tricky: they went out of production 10-15 years ago.
Sourcing 5¼ inch floppy disks, new or 'slightly used' is probably a greater challenge.

---

Some weeks ago a friend (AF) asked if I could copy a 5¼ inch floppy disk for him, leading to this little adventure...

AF had a 3½ inch floppy that he might have copied everything onto and wanted to check.
Without specialist equipment, I knew I wouldn't be able to recover all potentially readable data, but I offered to do what I could with a standard drive.

In the end, AF decided to try something else, so I  didn't get to do his copy.
But it did make me get my old 386-SX properly setup, networked so I could move files to/from it and a way to easily copy 5¼ in and 3½ floppy disks.
Before this, I still booted it occassionally, but the real-time clock battery had died and it had no network card or CD-ROM drive.

This decision led to weeks of farnarkling and some interesting lessons.

While it's unclear if my 386-SX will survive another 2 decades, the software can live on through tools like QEMU, WINE/Crossover and even DOSBOX. So there is some value in recovering the data both on the hard disk and my collection of floppies.
The impetus to recover data from unreadable media, 5¼in floppies, is obvious.
For the readable 3½in floppies, taking a copy now is a good investment of time if I ever want to access the data again: magnetic media does degrade over time. In another 20 years, the coating on those floppies may be flaking off in big lumps.

386-SX Initial Config:

• purchased late 1991, ~$3250
• 386-SX CPU, 20Mhz (selectable to 8Mhz). No FPU. 8Mhz ISA bus, 8-slots.
• Dual floppy drives, 5¼ in [boot] and 3½ in.
• IDE disk. 80Mb, WD AC280.
• single parallel port, dual serial ports, one used for mouse, other for modem.
• 5Mb RAM [max at time]
• Super VGA1024x768 16-col, 640x480, 256 col. 512Kb [K-i-l-o not Mb]
• 33cm display. [fixed scanrate, can be destroyed @ wrong Hz]
• mini-tower. pre-ATX power-supply (no 'soft' power switch or 'halt')
• No sound-card.
• DOS 5.0 and Windows 3.0

Current state-of-play is:

• 386-SX has DOS 6.20, Win 3.11, Networking, CD-ROM and Zip drive all working.
  Hard-drive cloned/backed up and (5) ZIP disks read and copied. [ZIP drive back in its box]
  Single 3½in floppy as A: drive. BIOS only allows booting from first drive.
• 5¼ in drive working as 2nd drive in a Linux machine (2001, Celeron 667) with built-in networking.
• Working through copying all  5¼ in floppies I can find.
  Tally so far [40]: 1 unreadable, 1 with errors.
  Update 04-Nov-2011: 210 floppies read, 40-50 5 in: 8 "no data". 150+  3½in, 5 "no data"
• Now have a USB 3½ in floppy drive, can read those at leisure on newer machines.

The most important lesson for me came about two weeks in:

• I was fixated on doing everything on my 1991 vintage 386-SX.
  At one point I was running through the options of replacing the motherboard and the various costs which weren't attractive given I was 'just playing'.
• Since USB became ubiquitous, finding machines/motherboards with floppy drive controllers is increasingly difficult, which means even embedded boards with FDC's are rare and expensive.
• Then I realised I already had everything I needed in the 2001 vintage Celeron system I had tucked away.
  It's loaded with Fedora Core 3 (support ended in 2004) with a linux 2.4 kernel. Old, but usable.
• It was perfect for what I wanted to do.
  It had a floppy drive controller and I could transplant the cable (with  5¼in 'slot' connectors) from the 386-SX to the Celeron 667.

 I also learnt a little about floppy drive connectors:

• 5¼in drives have a 'slot' (card-edge) connector on the drive and a header not unlike a 40-pin IDE connector (34-pin is used).
• 'Classic' 3½in drives use a socket connector similar to IDE connectors (but 34-pin)
• The $30 USB 3½in drive I bought uses an incompatible tiny connector (two versions, a plug and a socket with a conversion cable). Previously, I didn't know this variant existed. Noted to save other people from popping open put-together-permanently cases. I didn't care about the warranty, but the case needs to be firmly shut or drive operation is affected.

Before starting any work, I had to backup the original 3860-SX disk. My memory was that I'd bought a 30Mb 'RLL' drive (a 20Mb ST-506 drive with a modified controller).
Turns out I really had an 85Mb IDE (now called ATA or PATA), a "WD Caviar® AC280", not only larger, but it would allow me to connect an IDE CD-ROM drive in their as well. My unused hardware pile has any number of CD-ROM drives.

Most importantly, an IDE/ATA drive gave me the option of backing up the drive via an IDE/USB interface... Of which I have a number of versions.

I also had an old system backup of around 30 * 720kb 3½ floppies. DOS 5.0 and Win 3.0.
Using QEMU on a Linux system, I was able to restore this backup to a virtual disk drive.
Shuffling all those disks was painful and slow.

Connecting an early IDE drive to a modern(ish) IDE/USB interface didn't work.
Probably because additional commands were introduced to identify the drive, possibly because this old drive responded to "CHS" (cylinder-head-sector), not LBA (Logical Block Addressing). From the AC280 spec. sheet, the drive electronics did support any reasonable CHS settings, not only the physical layout.

My next preferred method was to connect a second IDE drive as a 'slave' (D: in DOS), fdisk and format it and copy the original drive contents, then connect this drive to a modern system with IDE/USB interface and back it up.

The first 3.5" HDD I tried from my unused pile (1Gb Fujitsu) had errors.
Next drive tried was a 3.5" 4.3Gb. Worked reliably.
In the final config, I replaced that drive with a slower, quieter 2Gb 2.5in drive, cloning it via an IDE/USB interface.

The Phoenix BIOS in the 386-SX is very old. Not only doesn't it support LBA drives, it seemed to limit drives to 1023 cylinders - and 15 heads/63 sectors. Around 470Mb.
A very large fraction of my time was spent fiddling with disk CHS specifications and attempting to get fdisk to ignore the BIOS settings.
[No, I can't update the firmware, the BIOS is pre "Flash-the-BIOS"].

I could setup multiple (large) partitions on the drive with Linux and the IDE/USB interface, but then would run into troubles under DOS and the 386-SX.
I tried 'fdisk' from DOS 5.0, 6.2, 6.22, Win-95 and FreeDos on the 386-SX, but all would only see the 470Mb allowed by the BIOS. Extra partitions would be displayed, but couldn't be changed.

I don't have enough unused 1.44Mb 3½in floppies to back up the entire 85Mb drive, so was very glad my 2nd method worked.

Getting a working ISA-bus (not PCI) network card was simple: I had two in my "unused bits pile".
The one I chose I'd bought new, a Netgear NE2000 clone, but I didn't realise that or find the box (with install floppy) for a while. Relied on Windows NE2000 driver and Internet downloads at first.
The other card didn't have a clear name/identifier, nor did I get a good match from the chip numbers.

This was another 'surprise': how to get any info on installed ISA cards.
I also spent 2 or more days fiddling with the IRQ/DMA settings on the NE2000 card. I'd forgotten the problems that PCI made go away. I ended up with IRQ 5 (COM2) and found an IO address range through trial-and-error. I don't know for sure if its a clash or not.
I couldn't find a tool that would list for me all the cards + settings in the system.
Norton's "SI" provides everything but DMA ports.
MSD (Microsoft Diagnostics) didn't help either.

Getting the CD-ROM working was a good idea, if a little problematic.
Using standard Linux tools, I was able to create an ISO image of the original DOS/Windows system and also add some additional tools.
The problematic part was creating a disk image with Uppercase filenames. Whilst DOS 6.2 (really MSCDEX) reads the root directory correctly, no files or directories can be read/listed.
Perhaps it is the 'Joliet' option I use that causes this... Had the same difficulty with the FreeDos CD.

I managed to get non-DOS booting via 3½in floppies:

• FreeDos [tried last]. Booted, read CD-ROM, not installed on HDD.
• BG-TLB: http://www.giannone.ch/bgtlb/ [Tiny Linux Boot]
• floppyfw - floppy firewall. http://www.zelow.no/floppyfw/

Until I tried it, I wasn't aware that FreeDos uses linux as a base. Uses 'SYSLINUX' as the boot loader and seems to have a kernel.sys.

It took a good deal of searching to find any Linux that would support:

• vanilla 386
• no FPU
• no RAM disk for under 12Mb,.

"Floppyfw" recognised the (single) NE2000-compatible network card, but didn't include a shell.
BG-TLB does include BusyBox, but only support 'plip' networking over the parallel port (not tried).
So while I have seen a linux shell prompt and been able to mount the DOS/FAT filesystem, I haven't been able to dual-boot the 386-SX or run it as a Linux only system.

Whilst FreeDos could read its install CD-ROM, DOS 6.2 was unable to read files/directories contents (the lowercase name problem above).

Another Linux not-tried was tomsrtbt: "Tom's floppy which has a root filesystem and is also bootable." http://www.toms.net/rb/tomsrtbt.FAQ. It advertises itself as "The most GNU/Linux on 1 floppy disk."
Which might have worked, but it formats 3½in floppies at the non-standard 1.7Mb, not 1.44Mb, with the caveat "Doing this may destroy your floppy drive".

One of the 'surprises' I got was being unable to replace the original 3½in floppy drive with a newer drive. The interface and connectors were all the same, but the newer drive wouldn't work in the old system.
Was it me connecting it incorrectly, a faulty drive or something more?
Unable to tell and unwilling to devote a bunch of time testing it.

One of the worst 'surprises' I got was after installing the IOmega (parallel-port) ZIP drive software on the system after a fresh install of Windows 3.11. [Windows 3.11 had decent Networking support and Microsoft still have downloadable a good TCP/IP stack.]
I had it all setup, tested and working and foolishly, in Windows, selected "Optimise settings" and the system hung.
Whereafter, the system couldn't see any Comms ports, serial or parallel. Which was very problematic because the 386-SX didn't come with a mouse port (DIN or PS/2). I used a serial port for the mouse.
Windows hung when it booted, leading me to try to revert the ZIP drive install and later to re-install Windows.
There were countless reboots and 6-8 hours later I gave up and went to bed.
I had realised/diagnosed that when the machine booted, the BIOS reported "0 serial ports" and "0 parallel ports". The BIOS setup screen only allowed me to selected HDD and floppy drive settings.

First thing in the morning, I powered on the machine and it worked perfectly. Including the original copy of Windows that would hang.
All I can think of was a power-cycle (off/on) cleared the fault, whereas a 'cold' or 'warm' reset (reset switch or ctrl-alt-del) didn't. In the many reboots, I hadn't thought to power-cycle the machine. [A note for myself and others experiencing weirdness on old hardware.]

I also have an old Dell Inspiron 7000, dating from 1999. I got it with a removable ZIP drive, figuring I could do backups and bulk-data transfers using it and the parallel-port ZIP drive. While tinkering around, I disassembled the other ZIP drive. It's an IDE/ATAPI drive, but the connectors are non-standard. I was hoping I'd be able to kludge it to work on another system - but not to be.

One of the BIOS limits, noted above, was it will only boot from the first floppy drive. When I'd configured the machine, I'd made the 5¼in drive "A:". Part of my reconfig was to move that drive and make the 3½in drive "A:", so it could boot from it. And most "floppy disk images" on the net are 1.44M for 3½in drives.
Then I executed a perfect "rookie mistake": I forgot to make a bootable 3½in system disk before moving the 5¼in drive. And all my system disks were, of course, 5¼in.

The 386-SX is in a "mini-tower" case with very limited space between the back of devices in the drive bays and the motherboard etc. This makes running cables and changing drives quite time consuming. Especially with older connectors that are loose and can be jiggled off. This was part of the reason to move to a 2½in HDD - much more space. I did need to find a spare 2½in-3½in mounting kit first.
I didn't believe the weight of the whole system, let alone just the removable cover. Meaty!

Early on I replaced the on-board/real-time clock battery so the system would remember the time over reboots. More modern systems use 3V lithium batteries (CR2032) for this. This old motherboard used a 4.5V alkaline battery (mounted off-board with Velcro). About a week in, I used 3 AAA alkaline batteries in a modified carrier (and a cannibalised connector) to craft a replacement. A less pretty way is to load 3 batteries in a cut-to-length tube with wire soldered directly to exposed battery ends. Soldering wires directly onto batteries requires some technique. You may need help if you try it. There could be an explosion risk with alkaline batteries becoming overheated (they are marked "do not dispose of in fire"). Research this properly or get help if you choose to do this.

The system, including DOS, quite happily accepts dates of 2011. No problems there...

One of the problems I haven't addressed yet is: "How do I clean the drive heads?"
Back in the day when I was a sometime 'operator' on mainframes, cleaning tape drive heads was part of the ritual. We used Isopropyl alcohol + cotton swabs - because it didn't leave a residue. The swabs would always come away stained with oxide coating from the tape.
For these old drives, I've two reasons to want to be able to clean the heads:
a) these are old drives and may well have an internal dust build-up, and
b) older disks are likely to shed more of their coating than when new.
Some media formulations from the late 1980's are known to suffer problems. I've heard first-hand accounts of the work needed to recover period audio-tape recordings due to this problem.

• 5¼ inch floppy drives load the heads directly in-line with feed-slot.
  It is possible to get a swab in there, though v. difficult to see what's happening.
• 3½in floppy drives drop the disk to both lock the disk in-place and load the heads, hence the heads, being offset, aren't accessible for swabbing through the feed slot.

Another thing to investigate...

What is an electronic document and why that Matters.

#comment-2157 agimo.govspace.gov.au back-to-the-future-another-chance-to-influence-coe-development

#comment-2158 agimo.govspace.gov.au back-to-the-future-another-chance-to-influence-coe-development

First one says:
 There isn't yet a clear Industry Standard, so do Nothing for a while
longer.

Second says:
 The basis of your problem is you're conflating many problems with
conflicting requirements under the banner of "document exchange".
 And there's *never* a good reason to give non-authors an editable copy.

With a little postscript: FOSS definitively solved the "multiple
concurrent update versioning problem" - 20 years ago?
I didn't say, the 'secret sauce' is Use Simple Text Files!

That's at the heart of all these problems: binary formats.
Should've been banned 30 years ago... But weren't and now we're paying
for it.

Feedback model of Software Production

The bottom part of the diagram is a representation of a single step in the software production cycle.
It consists of two Amplifiers (A1, A2) with gains G1 and G2.
[The top part is an outline of the possible succession of steps in the process. This too has interesting characteristics. Not addressed here.]

Part of the output of A1 leaks back, via A2, into the input of A1.
A classic schematic for a wide-band amplifier or an oscillator.

How much is fed back, and in what time-relationship to the input?
The value "R". (Really it should be "Z" for impedance, and have lead/lag/in-phase components)

The crucial aspect of the diagram is that all inputs are "positive" - if the output directly follows the input, not the inverse (as found in a classic "negative feedback", or correcting, system.)

The model says there are only 3 internal metrics to be measured an one external inflow and two outflows.
The 3 internal metrics:

• G1 - Raw Lines of Code per Day
• R - errors per hundred Lines of Code
• G2 - Time to fix one error. In a normal Project, this can't be less than 1/2 day because of overheads, analysis and diagnosis, testing and change commit.

The external inflow is Errors from steps further along the chain.
I've forgotten to draw its inverse, an outflow to previous steps.

The most obvious outflow is "Delivered Lines of Code".
You can see from the nature of "R", there may be significant 'lag' (time delay) so at notional Project Delivery, Errors may not be fully realised.

There are exactly 3 dynamic states of this system, given that G1, G2 and R are constant.
The "gain around the loop", G = G1 * ( G2 * R ).
[if r = 1/R; then G = G1 * (G2 / r ). Possibly more intuitive.]

The Loop Gain, G, can be greater than, equal to, or less than 1. [mistake in diagram, I wrote '0' (zero)]
The system dynamic are:

• G < 1. System "converges". It heads towards the X-axis asymptotically (?). The smaller G, the faster the convergence.
• G = 1. System is in a "holding pattern". It never moves towards completion, but doesn't move away.
• G > 1. The System "diverges". Every day of effort produces more than one day's extra effort. The classic "Death March".

Having 3 variables, they can be manipulated to push the system in a desired direction. If you were paying for the project and wanted a result, you'd want the smallest values for R and G2.

• Reduce G1. Slow the coding process. Counter-intuitive and probably would cause the Project Managers' head to explode. But its there in the figures. Spend more time fixing errors and less in creating more new code and their concomitant errors.
• Reduce G2. Better people, better systems, better tools, better training to find faults faster and produce better solutions. So do subject-area experts. Lesson: Don't move people away from what they know.
• Reduce R. Better Design, Design and Code Reviews and better people/tools/processes for coding. Create less rework, adopt practices that reduce errors (like pair-programming). And most crucially, don't over-work people. Even a small increase in Error rate can push a project into the Red Zone...

Programming Projects are defined by Rework and its management.

Projects do not "fail/get later one day at a time".  They fail before they start.
Let go, they often gallop away from a solution.
No amount of effort can retrieve them - the more effort expended the more that's needed to complete them. This was the genesis of Microsoft's 2005 "Longhorn Reset". 25,000 person-years were discarded.

Wireless Broadband Plans (Australia)

Talking to a friend about "going broadband" led me to think I should write something down...

If you are "on the road", MacDonalds offer free WiFi hotspots. If that works for you, saves all the hoopla of getting 3G broadband.
'Lifehacker' suggests there is a 50Mb download limit.
maccasfreewifi.com provides a hotspot finder for 706 locations.

Additional Comments [19-Jan-2010].
This writeup of George Bray's is clear and concise.
I think it shows a definitive, cheap, simple solution to Mobile Broadband, plus flags a bunch of caveats.
Stop now and go read his piece if you want or need to actually do something soon.

Here's a writeup and video of my "Broadband from the Beach" equipment.

http://geobray.com/2010/01/19/broadband-3g-from-the-beach/

George
  

Caveat:

Not having had/run a mobile wireless broadband service myself, there'll be stuff I don't know about...
Please contact me by email or "comment" (below) to correct errors/misinformation, provide links or additional war-stories/advice/information/tips/comments.

Mobile Wireless Broadband:

There is some confusion about "Mobile" and "Wireless" Internet/Broadband.
There are mobile-phone 3G networks, long-distance data-over-wireless and 802.11 Wireless Ethernet, or "WiFi". This piece concentrates on 3G-based services. Australia is yet to see broadscale WiMax or 802.11 networks.

Mobile or Wireless Internet/Broadband and Laptops are an extremely powerful combination.
The only caveat is Mobile Internet generally comes from a USB device (or via USB from your phone).
The lack of a hardware/stand-alone firewall means it's difficult to share a connection and you give up some security.

Hand-crafted arrangements can be built:

... in Armidale and the only place I could get reliable 3G link was from my bedroom window.
Nokia N95 in the window with bluetooth link to EeePC 701SD on the desk and from there an Ad-hoc WiFi link providing access to me anywhere in the house.

Whirlpool Forums (a.k.a. "Broadband Choice") are a very good resource: http://forums.whirlpool.net.au/forum/114
Whirlpool have a list of Networks and Resellers, as of Dec-2009.

There are three "3G" networks in Australia offering data (HSDPA: High-Speed Downlink Packet Access) and competing reasonably aggressively:

• Telstra - best coverage, generally best potential speed. Max 42Mbps,
  
• Optus
• Vodafone, 3, Hutchinson

"3G" (3rd Generation) mobile phone services are based on CDMA (Code Division Multiple Access) protocols. This means if you have signal, you can get service (not so with GSM, which is Time Division Multiple Access). It also means you can improve your download speeds and effective coverage by attaching a larger aerial.

Telstra have very few or no resellers of its NextG broadband network.

Optus and Vodafone/Hutchinson have many resellers: folks like Virgin Mobile and Exetel offer great packages using one of these networks.


But there is a nasty 'gotcha': "Roaming Charges".
On some plans, especially low-end, you can be charged (from memory) around $100/Gb for roaming.
This may have changed, but is worth checking for specifically.

"Roaming" is where your provider doesn't have a service, but another provider does and will carry your bits - for a price...

On older 2G ("GSM" or 'digital mobile') networks, the data is called GPRS - General Packet Radio Service and used to be quite expensive and comparatively slow. Some carriers use this as a fall-back when you are 'out of area'.

Some friends did the "Gray Nomad" thing through Western Australia and back to Sydney this year.
They got a Telstra 3G USB modem and a new laptop.
It generally worked very well. They had about a week of 'glitches' at the beginning, but eventually found a helpdesk operator who'd seen the problem and got them on-line.
They also had a problem with the laptop power-supply 'failing'. It ended up being a seed lodged in the connector.

Travelling, they had 3 types of service:

• None.
• Slow, out in farming communities.
  
• Fast, near mining towns and cities.
  

They were able to stay in-touch without on-site support: that's pretty impressive.
George notes on the CLUG list he regularly achieves 4Mbps with a tripod-mounted directional aerial.

Google threw up a few sites offering service comparisons:

• broadbandguide.com.au
  
• comparebroadband.com.au 
• broadbandexpert.com.au (Seems to be limited, perhaps sponsored)

They have different coverage. For example, only 'compare broadband' threw up  Exetel plans for $17.50/month for 1Gb and they only say "speeds vary".

Selecting your plan:
[See also below "Specific advice for selecting a service and setting up your connection"]

• Choose Prepaid or Contract. Prepaid means you can experiment.
• Contracts: You are locked in for 12-24 months with "Early Termination" penalties. Beware of "automatic renewal" at the end of the contract - they may require you to specifically "opt out".
  
• Prepaid: You need to look for $ per Gb and 'Expiry' - how long you've got to use a payment. (Hint: Don't apply payments until close to when you need them of you might lose them.)
• Prepaid 'gotcha's: Recharges must be planned.

1. First problem is to change plan options (phone or internet mobile plans) before applying the 'recharge'.  Payments/recharges cannot be altered once applied. [From my limited experience]
2. It can take 48 hours for a 'recharge' to be applied to your account for some carriers. While you might technically be in credit, your service could stop or be charged at a (much) higher rate until the Telco systems process your payment.
   

• Modems: Are they 'locked' to a network/ISP, or can you use it anywhere? You're also going to be interested if the modem is supported by your Operating System.
  Important user advice: Select your modem/gateway first.
  
• Speed and Download limits: Mostly how plans are differentiated.
  Beware of the costs of downloading e-mail.
• Monitoring Usage: Whether on a pre-paid or contract plan, you will most likely have to monitor the usage seen by your provider. George in his CLUG post, points out that that there is no standard way and potentially is an application tied to a specific operating system.  There is also the issue of stale and incorrect usage data being reported by the tool.
  This is a nasty 'gotcha': If you are paying by the Mb and can't reliably track it, don't fly too close to the limit. At best you'll be suddenly cut-off or 'traffic shaped', at worst get a very nasty bill.
  

Virgin Mobile (and presumably others) also provide a Home Phone package.

For $80/month (Dec-2009) Virgin Mobile give you a box that connects via the Optus 3G network and gives you unlimited VoIP calls within Australia (with a normal dial-in number), 4Gb/mth download and they're a little coy about the speed. But it falls back to 128kbps.

People like Virgin (and Optus?) have a "Fair Use" policy.  If they deem your usage 'excessive', your account may be cancelled or you'll incur fines.
'gotcha': On the CLUG list there was a report of Exetel dumping a customer as they "were unprofitable". You can't avoid this: the service providers supply you their service subject to their Terms and Conditions. You must agree to the "T&C" or there is no contract.

The only 'remedy' is to avoid the problem. If you intend to heavily use a service, sign-up for an appropriate plan.

Great Tips:
Tridge happened to mention at a CLUG meeting that at home he runs both an ADSL link and a back-up Mobile Wireless link. If ADSL goes out, he can still access his home computers from outside.

The normal Unix "route" software for IP doesn't allow for asymmetric links: there can only be one default route.
Packets may find their way to your host via a second route, but must return by the default route.

The details of "asymmetric routing" using "iproute2" are spelled out in this thread on the CLUG list.
Links to iproute2 Project and Linux Advanced Routing and Traffic HOWTO.

Speed and Latency:
"Oils ain't Oils" applies.
 4Mbps on mobile wireless will not provide the same apparent level of service as 4Mbps over fixed-wireless, 802.11 or fixed-line (ADSL or Cable).
Whilst you may have a theoretical download speed of 4Mbps, TCP/IP needs to correct errors and to wait to 'ACK' packets to come back before sending more. The round-trip-time can limit link performance.

Mobile wireless suffers from transmission errors (noise + lost packets), variable data rates (there are other users competing for air-time) and long latency (affects round-trip-time) due to many layers of packetisation, buffering and forwarding.

Long latency only marginally affects big downloads (the TCP/IP 'ACK' problem), but can be a deal-breaker for real-time services (VoIP, Skype, IM, ...) and when downloading many small files - as in normal websurfing and collecting email.

Radio reception is fickle for many reasons - tune in to AM radio at night for an example.
Mobile networks necessarily need to track 'handsets' and SIM cards to route traffic: what towers are you currently near?

Which means mobile networks must have a centralised handset location database and a single or limited number of interconnect points.  This means your phone or USB dongle is many hops from 'central office'. The ISP connects from there.

Telstra trumpet they support 21Mbps (going to 42Mbps?) - but not on all towers and your "actual speed" will vary with aerial, equipment, signal strength, other users and backhaul capacity...
As my Gray-Nomad friends found, Telstra won't invest in unwarranted (unprofitable?) capacity.

Even when you hit a "white spot" (high-speed tower), you have to compete with everyone else in the area for a slice of bandwidth.  May be great in the bush, may be less great in highly used areas - like the city or a rural/regional tower offering the only effective broadband in the area.

Additional Comments:
[30-Dec-2009] Thanks to CLUGers, in no particular order, for their help and comments:

• Alex Satrapa
• Scott Ferguson
• Ian Munsie
• Keith Goggin
• Miles Goodhew
• George Bray

A great piece on using mobile wireless by George Bray sent on the CLUG list.


A note on the CLUG list on setting up Telstra 3G (the MF626) on Linux (Ubuntu 9.04) by Carlo Hamalainen. [19-Feb-2010]

Contract 'gotchas':

"Early Termination Fees" that most providers are charging these days:
Sign up for a two year contract costing $480 all up, they'll sting you for $540 if you terminate the contract before the two years is up (not to mention, they automatically sign you up for a further two years if you don't opt out during the three month grace period at the end of the contract).

The moral of the story is make sure you check the conditions on those contracts, folks.  Early termination, "roaming" (in the case of at least one provider, they will charge you the 'normal rate' for data if you fall off the 3G network onto their GPRS network), "acceptable usage" and other caveats.

General comments on Mobile vs Fixed-line service:

Summary: "it sucks, don't do it."  :P

Seriously, Virgin Mobile offers free tethering with their $54/month iPhone plan - that's $54/month including payments on the phone. I don't have the details on me, and I can't find the plan on the Virgin Mobile website, (the kiosk in the Canberra Centre near Mac1 will be able to help you).
I think it's a 300MB/month data allowance.

On the flip side, I've never had a good experience with 3G broadband except for the situation where you can mount the antenna in just the right spot and leave it there.
In other words, you get the best 3G coverage in the same places you get the best ADSL 2+ coverage.

Optus specific comments:

• provide an internet addressable IP to their mobile broadband customers
• does not charge extra for roaming onto their slower GPRS link
• provides a "usage meter" to help you track your usage (some reports of delays and misreporting)
• Virgin use the Optus network with mostly the same Terms and Conditions, with the additional proviso that peer-to-peer traffic may be capped.

Non-mobile wireless, Canberra only:

Netspeed 'Longreach'  works with Linux (SuSE 11.2) with a 2km range, but "Slows when Foggy". The Netspeed whitepaper says it's a fixed wireless network using a Motorola product with a 10km range. Expect 7Mbps, with a 14Mbps maximum.  Their 'availability' checker lists 3 deployed base stations (Canberra South Side) and 2 towers under deployment (north side).  Good rates if it works for you.

Tethered data on the Hutchison/Three network:

I use my Nokia 6110N paired via Bluetooth to my netbook for tethered Internet access. I pay $8/mo for the privilege of the "Internet" connection (Otherwise you're in a walled garden). They have a $5/mo plan with much higher data cost and lower included data volume. Their roaming data charges used to be astronomical, but I think they changed this recently ...
Generally it's slow, the latency's massive, but it works and it's very convenient for email/web. Also, if you commit to a 12 or 24-month plan they double your included data volume.

Specific advice for selecting a service and setting up your connection:

When shopping for a 3G plan I would recommend selecting a modem first, then finding an ISP that will sell you it - or will sell you a SIM only.

One feature you should look for is an external antennae connection - very useful in some areas. Check to make sure the model fits without interfering with access to other ports, and that it doesn't raise to laptop off the desk.
I'd recommend a 5m USB repeater cable for better reception, and to allow placing the antennae away from the user.

Make sure your machine is capable of supplying more than 500ma to the USB ports. Consider buying a "split-tail" USB lead - one input for data and power, another for extra power from a second USB port if your machine is USB power taxed or lacking. For about $3 from JayCar you can buy a portable USB power supply - it takes 4 AA batteries and reduces the output to 5V, hack on another single AA carrier and it will supply 5V from Ni-MH batteries.

Most of the UMTS (3G) modems I've looked at have the option of selecting HPSDA only (no roaming surcharges). I'd recomment setting your modem to that unless you are going into a bad signal area. Some plans do not include SMS.

Check the modem model numbers - when I was looking a couple of months ago I noticed that Dick Smith in Civic had 4 prepaid Dodo Huawei modems.
I noticed that one of them had an extra alpha in the model number - one sales rep assured me the "G" model was the same as the non-G, another said it was better. A quick search on the internet showed the "G" as a lower powered model - supposedly no longer distributed. The Huawei has an external antennae connection.

You will hear reports of slow connection speeds under Linux with some modems - generally because the user has an old driver module with a very small maxSize (the module takes a vendor and product parameter only).

Gateway, connection sharing and stand-alone firewall:

... have a bigpond 7.2 Home Network Gateway (3G9WB)

• 4 port Ethernet router
• configurable security
• runs off 12V DC with internal voltage regulator more than adequate for car use
• can use high gain antenna for fringe area
• EXPENSIVE $90 per month 5GB shaped
• VOIP works OK mostly @ ~40kms near line of sight to tower (or better

So I bought a new Intel Mac Mini...

In early 2006 I bought a Power PC (PPC, G4, DDR2-SODIM) Mac Mini to use for a Uni course.
Could've gone Intel, but it was new technology, so I avoided it...

Coming up to 4 years on, and I've filled the disk, the CPU fan is a little noisy, new things like 'Google Chrome' are 'intel-only' and OS/X 10.6 ('Snow Leopard') is Intel only... Plus VMware and Parallels can run on Intel platforms.


This time I paid the ~20% extra for a top of the line 2.53Ghz core duo, 4Gb DDR3 RAM, 320Gb 2.5" SATA - which is roughly 4 times the capacity in all dimensions as the PPC version.

First surprise, doesn't come with a DVI-VGA adaptor in the box. BUT, now there are 2 video connectors, 'mini-DVI' and 'miniPort'. The shop only had a 'miniPort' VGA adaptor on the day and I went home with that...

Attach to KVM, boot and - no monitor found. VGA adaptor works fine directly connected.

Second surprise, 800Mhz Firewire is a 9-pin connector not 6-pin (400Mhz on PPC Mac).

Booted new Mac and it says "got an old Mac?" and gives instructions to reboot the old one "holding down the 'T' key" to start a transfer. Which meant closing out all my running Apps and discovering just what they meant.

Which was hold down the 'T' key whilst the machine was shutting down, before the reboot.
It comes up in 'Transfer mode' - displaying a "firewire transfer" symbol. Apparently you can use an 'ethernet' transfer mode... Didn't try that.

After 2-2.5 hours, the transfer completed - 75Gb moved over. Accounts, Preferences, Applications and User Data copied and working.

I've had to reload Adobe Acrobat Reader, apparently it wasn't a 'Universal Binary'.

Even the licensed & registered 3rd-party software I had (MS-Office, NovaMind, OmniGraffle) 'Just worked' on reboot. The transfer was clever enough to not move system binaries. As a bonus, I got a work Perl system back... In the past I'd installed a second, non-Apple, version and managed to zap CPAN.pm on both versions...

Third surprise was initialising "Time Machine". It took 2.5hrs to transfer 75Gb, which is about right - I've timed the old HDD @ ~20Mb/sec sustained raw read rate.

It took a day and a half (!!) for "Time Machine" to initialise... 33-36 hours, Ouch!
The USB drive, a 1Tb Western Digital 'Book for Mac', runs at about 10Mb/sec for raw reads (36Gb/hour).

I can only guess why it ran so slowly... The "Time Machine" is a HFS filesystem.
"Activity Monitor" showed some very large numbers for Gb read from the system Disk, but the amount written to the USB drive matched the progress numbers reported by 'backupd'.

In the middle of the initialise, I rebooted the machine because I could see no evidence of progress.
Rebooted, restarted the initialise and off it went again. Seemed to pick up from where it left off.

There were one or two other times that it took hours, literally, to backup a few Gb.
Now it does 2-3Gb in 5-10 minutes, as you might expect.

What I love about "Time Machine" is this is the first time I've had a good backup system for my home desktop. Seems odd as this is what I do professionally, but there it is.

Another cute thing with "Time Machine" is it can use a Network connected device - like a wireless connected Apple 'Time Capsule' or instructions are out on the web for using a "NAS" - like an old Linux box.

The "Time Machine" interface is well thought out.
You 'enter Time Machine' and the Finder comes 'front and centre', the background changes to 'beginning of the Universe' picture, all the other Apps slide away and the rest of the decorations appear.
Neato!

When you're done, your desktop just slides back into place. Very cute.

There's a major caveat with "Time Machine" - it's granularity is "whole file".
I use 'mailbox' format (a 'folder' contains all messages in a single file) versus 'maildir' format used by Apple Mail (every message is one or more files in a directory). Everytime a new message gets added to one of my 'folders', the entire file is written. I've looked and there are several which as 1+Gb :-(

It's pretty much 'rsync' with a nice GUI and a few tricks in the HFS filesystem.  One of which I read (to confirm) is that it allows directories to be linked.  This was removed from POSIX because you can get complex loops in filesytems...

Things I don't like about "Time Machine", but aren't reasons to bail:

•  I haven't been able to find how to turn on decent logging.
  The default syslog entries written in /var/log/system are high-level (#files, total size).
  Where/How do I get a list of the files it dumped?
  
• I could exclude Thunderbird's mail folders and save on backup time and space.
  What I'd really like if I could tell it that these files grow by concatenation, and to link to just the most recent version... Though data might be lost if file is truncated, but the semantics of that should be simple. Eg. if the current backup version is perfect subset of the new file, just concatenate the new lines. Otherwise, create a copy.
  
• Haven't found, after a quick look, the API. Be nice to be able to integrate with 'rsync'.

'locate' works, but there is some internal magic that seems to exclude 'Desktop' files...
Almost everything is scripts, but I can't find the list of excludes. Perhaps it is just silently failing because of the number of files I have in a directory there...

[30-Dec-2009]
The Western Digital "MyBook" only transfers 10Mb/sec (36Gb/hr). Seems slow.
If "Time Machine" reads or writes every block on the filesystem, that would explain the initial load time and observed volumes of data read.

Mini-display port versus Mini-DVI.
If two VGA adaptors are used, Display-port is the 'primary' (system boot and menus).
If VGA and DVI on Mini-DVI: the Mini-DVI is the 'primary'.
[Don't have enough dongles to test all combos]

Working with a USB/DVI KVM.
The Intel Mac Mini has a DVI-D connector, the PPC Mac Mini has a DVI-I (integrated, allows for DVI-D or DVI-A).
The connectors on the KVM I bought are DVI-I, which won't plug into a DVI-D socket. (A DVI-D cable will plug into either a DVI-I or DVI-D socket).
I won't spring $50 for a separate DVI-D cable  until I talk to the vendor.

Meanwhile, I saw a $25 VGA to DVI-A cable.
Connects to the KVM, but nothing displays. Presumably because the DVI-D connection on the monitor only does digitial. Who'd have thought :-)

The PPC Mac Mini came with a VGA/DVI-I adaptor. It doesn't work in reverse. :-)

The KVM exhibits the usual "doesn't play well with sleeping computers" problems:

• While the KVM does connect speakers/microphone through, there is sporadic hum.
  
• with 2 or 4 ports connected, put a system to sleep via menu and it is 'woken' after a second or two.  Something to do with the USB  interface of the KVM. Execute a delayed command, flip to another input - All Fine.
• Single device connected - system will sleep from menu, but at some random time, it's alive again.
• Waking a system is impossible without unplugging/replugging its USB connection to the KVM
• Not 'sleep' exactly, The keyboard and mouse sometimes become unresponsive. Either reconnect them to the KVM or reset the KVM (remove all powered connections, including video).

This DVI/USB KVM has the habit of complaining about selecting an inactive port. It will lock up (needs 'reset') if moved to an inactive port. Not sure if that's sometimes or if there on the port for more than a few seconds.

I tried turning off the 'beep' from the keyboard - locked up & needed a full 'reset'. Could've been I swapped it to an inactive port.

DDR3: Memories ain't Memories

Purpose: Summary/Intro to DDR3 memory with Intel Nehalem (iCore 7, Xeon 3400, 3500, 5500, 7500). NOT a reference document. Dell biased. [Nov 2009]

Audience:
Those needing to configure memory sub-systems on Intel Nehalem/DDR3 systems:

• Selecting between pre-configured systems.
• Populating DDR3 memory on a new motherboard
• Upgrading an existing system
• Configuring a new server
  

Background: I got into trouble specifying some new Dell servers. DDR3 is different enough that you will most likely not get the result you intend.  This is about Intel, not AMD, processors though some concepts will transfer.

Selecting a DDR3 memory sub-system config requires knowing what you want to Optimise/Maximise:

Speed (Latency or Throughput) vs RAM Size vs Price vs Power vs H/A features

This note is skewed to Dell equipment, read their "DDR3 Technical Whitepaper" for Poweredge 11g servers" (PDF).
The Intel Xeon 3400 series architecture is described in this Intel PDF.
 
Dell's "Quick Reference Guide" for Installing and Upgrading DDR3 Memory in 11g servers contains an excellent summary of memory options for 5500 series processors. [There is an error in the last diagram. Text says 24Gb, diagram shows 36Gb]



Dell's "Transition Guide" for 11th Generation servers, (PDF), page 11, outlines their DDR3 memory restrictions by type, speed and ranks per MCH.


Reading the Intel doco, these acronyms were unclear to me:
UP = UniProcessor, DP = Dual Processor, MP = Multi Processor

Wikipedia has a good page on "DDR3 SDRAM", lots of technical info.
Other of their articles describe  "ranks" and the relationship to x4 and x8 (4-bit or 8-bit DIMM's): DDR_SDRAM, and DIMM#Ranking.
Dual-rank (DR) DDR3 is made with x8, Single-rank (SR) with x4 DIMM's.
Quad-rank (QR) modules exist, not sure what organisation.

Basic DDR3 data:
1Gb to 16Gb DIMM's possible. 1,2,4,8 Gb sold by Dell.
speed (Mhz) in 266Mhz steps: 800, 1066, 1333, ?1600? (not available from Dell in 2009)
64-bit transfers. 72-bits transferred for ECC.
800 = 10 ns,   6.4 GiB/s per channel (1,000,000,000 per Gi) B = bytes.
1066 = 7.5ns,  8.533 GiB/s
1333 = 6.0ns, 10.667 GiB/s
1600 = 5.0ns,  12.8 GiB/s

Intel describe Nehalem as a 'tock' (new architecture) in 45nm. A 'tick' (more speed) coming next.

32nm variants coming later.

Wikipedia on Intel Nehalem microarchitecture:
on-chip Memory Controller Hubs (MCH).
DDR3 memory or DDR2 "Fully Buffered" some chips.
No Northbridge chip on 'advanced' chips, with Southbridge collapsed into PCH (Peripheral Controller Hub).
QPI (Quick Path Interconnect) for NUMA: access non-local memory in multi-chip systems.
 Replaces FSB on high-end.

Wikipedia summary of the Xeon 5500 series (DP not UP). Note the 'basic', 'standard', 'advanced' ranges. DDR3 and QPI speeds change as well as CPU clock speed through the range.

There is an obvious throughput limiter: the common 8Mb L3-cache and the MCH interface.
Presumably it's "non-blocking". i.e for 3*MCH @ 1333Mhz, around 32Gb/sec (Intel X5570).


DDR3 memory sub-system Variables:

Prices and Sizes:
faster DIMM's are more expensive.
larger DIMM's are more expensive per Gb.
8Gb is largest from Dell and premium priced.
Registered DIMM's are more expensive as well as use more power.
Different sized DIMM's can be mixed - but same size & rank in every corresponding slot.

Speeds:
DIMM speed - not the whole story. Fastest bus speed possible.
Bus Speed. All buses clock at same rate, including with Dual Processor.
Processor Memory Controller Hub speed (MCH) - may be limited.
 5500 series (DP) has 3 MCH + 2 QPI links, but varies from 800Mhz to 1333
 3400 series (UP) has 2 MCH at 1333Mhz. May only support 2 slots/MCH.

NUMA:
Memory is tied to a chip and non-local memory is accessed across the QPI (direct MCH to MCH).
Without a second processor, only one set of DDR3 slots is accessible, for a DP board, halves the available slots.
Processors on a DP board have to be identical. Can't mix'n'match different models.

Slots:
Max DIMM Slots/channel: 3.
 Some processors only support
Total DIMMs: Processors * MCH * slots/MCH.
  Some  boards only supply 2 slots/MCH.
  Lower-end motherboards may only provide 2 of 3 available MCH
slot 0 - furtherest from MCH (?). Must be populated first. Implies geographic addressing.
Dell derates maximum bus speed:
 - slot 0: 1333
 - slot 1: 1066
 - slot 2:  800Mhz
I.e. if anywhere on the bus there is a chip plugged into any 'slot 2', the bus is limited to 800Mhz.

Ranks:
ranks - roughly 'chip select' or 'bank select'. A type of Interleaving.
 up to 8 ranks per Channel.
DIMM's come in 1-, 2-, 4-rank: SR, DR, QR
 single QR allowed (Dell), slot 0 only.
Diff ranked DIMM's can be mixed - but same size & rank in every corresponding slot.
 Dell don't supply SR or QR DIMM's as a server option. (2009, appears that way)

DRAM chips:
x4, x8 - DRAM chip organisation. 4- and 8-bit chips.
x4 is SR.
x8 for DR. and QR is ?

DIMM Types:
Unbuffered vs Registered (UDIMM, RDIMM)
Parity vs ECC
address parity only on RDIMM's.
RDIMM's use 1Watt more per DIMM.
Can't mix DIMM types on a bus?

Other:
Other memory configs available: Mirror, Lock-step and SDDC (Advanced ECC)

• "Mirror" mode - 2 DIMM's per MCH, but in RAID 1 config.
    For high reliability. Even with ECC, caters for dead chips/connection.
    reports half installed memory to O/S.

• Advanced ECC mode - ganged MCH's. 128-bit fetches.
  Implements SDCC - Single Device Data Correction for x8 DIMM's

Dell and Intel recommend "Balanced Configurations" and "Memory Optimised" modes.

Standard Memory organisations from Dell are:

• High Performance (Slot 0 only used)
  
• Balanced Performance
  
• High Capacity (all slots populated)
  
• Mirror Mode or Advanced ECC
• Power Conscious

Intel Xeon Nehalem processor ranges:

• 3400 ("Lynnefield") UP
  
• 3500 ("Bloomfield") DP
• 5500 ("Gainestown") DP
  
• 7500 ("Beckton"?) MP. Not released (Q4-2009)
  

Notes:

The FSB has gone and is now replaced by Memory Controller Hubs (MCH) and QuickPath Interconnect (QPI). The word 'interleaving' is not used.

Each MCH can control just 3 DIMM's. But Dell's DDR3 bus clocks slower as you add more DIMM's... 1333 for 1 DIMM, 1066 for 2, 800Mhz for 3 DIMM's.
There are Single-, Dual- and Quad-rank DIMM's. Max 8 ranks per MCH.
[Connection charts exist that visualise the rules]

The QPI allows a chip to access memory via off-chip MCH's.
Of course, if you only install one processor chip in a two-proc board,half the DIMM slots are unusable. Need the MCH's of the 2nd chip to get to them.

RDIMM's use more power, but can be larger, faster and support 'address parity' and are required for more esoteric modes.

The Xeon 3400 series has 2 MCH's and often only 2 DIMM's per MCH on the motherboard.

The 5500 series has 3 MCH per chip. Some motherboards have only 2 DIMM's/MCH.
For a two-chip system, max 18-slots - with the caveat only 9 accessible if a single-chip is used.

Upshot:

• If you want 'cheap', you'll use UDIMM's@1066
• If you want 'fast', you'll use 1 RDIMM@1333 per MCH and a CPU to match.
• For 'max memory' - you're down to 800MHz and fill all slots.
• for high reliability, you might use 'Mirror mode' or SDCC.

Speed: Throughput vs Latency:
There is around a 20% memory throughput improvement by going with 1333Mhz memory where possible.

This piece on a SUN blog has a table of memory throughputs for Intel Xeon 5500 series with DDR3:

• "Table 3 - Relative Bandwidth Comparisons"
• "Table 4 - Relative DIMM Power Comparisons"

It lines up ranks, bus speed, DIMM speed and number of channels populated against 'throughput'.
[Not access speed. High-performance mode is 1*DIMM per channel]


I've not been able to get clear info of the effect of DDR3 "ranks" on performance.
From the tables in the SUN piece, it seems to imply that memory controllers understand 'ranks' and can utilise them to improve throughput (not latency).

Another caveat is that SUN claim to be able to run their memory bus @ 1333Mhz even if 2 or 3 slots per channel are filled. The generic and Dell doco I've found says the bus derates to 1066Mhz for 2 slots and 800Mhz for 3 slots. 

SUN don't give the bus throughput for "100%". A snippet from their table.
 
1xDR @ 1066 = 36%
1xDR @ 1333 = 44%
2*DR @ 1066 = 68%
3xDR @ 800  = 74%

Quotes from the SUN conclusions:

Key takeaways from the above (Table 3 on bandwidth) are:

• for DIMM configurations that support both speeds, memory bandwidth is up to 12% higher with 1333 DIMMs than with 1066 DIMMs (which is not obvious from chart since it shows bandwidths relative to 1x DR per channel and not a direct comparison between 1333 and 1066 for a given configuration)
• for a given capacity, one dual rank DIMM per channel provides higher bandwidth performance than two single rank DIMMs per channel

and

From the table (4: Power) it can be determined that:

• for a particular DIMM configuration and bus speed, DDR3-1333 DIMMs consume up to 6% less power than DDR3-1066 modules
• for a given DIMM configuration, the incremental power required to operate DDR3-1333 DIMMs at 1333MT/s data rate vs. 1066MT/s is 8% or less (also not obvious from chart since it shows power relative to 1x DR per channel and not a direct comparison between 1333 and 1066 for a given configuration)

and somewhat less obvious but equally important:

• a dual rank DIMM operating at 1333MT/s consumes less power than two single rank DIMMs at 1066MT/s

My Life as an Inventor

I always carry a notebook and have filled up more than I recollect... Ideas and invention has always been a fascination for me.

My work in I.T./Communications - Systems and programming - have always been an outlet for my creativity.

1972-1979

Study at Uni of NSW (Computing Science and some Chemical Engineering), 2 years as Analytical Chemist plus 2 years in mainframe Operations at CSR, 2 years full-time study plus clerical and Computing work.

One of the defining events for me at Uni was being taught "Operating Systems" by John Lions. I was on the second ever Unix kernel course, the first with The Book: the Lions Commentary.  As a matter of passing usefulness, we had to learn 'C' in ~4 weeks.

1979-1984

Worked in I.T./Computer in Communications and National Security.
Demonstrated good design and troubleshooting skills.
Worked on projects, real-time systems maintenance, database development and high-performance/high-availability systems.
Ran the International  email service for O.T.C.

1986
At Softway, with Greg Rose, one of the principals, I put the following 4 ideas to the board:

• RAID - we were a Unix systems and kernel house and could write drivers.
  
• Multi-protocol Routers as an extension to the CSIRO micro-node we were programming
• Multi-function printer/fax/copier
• Digital photocopier/printer

The ideas were turned down without comment. 3 out of 4 went on to become major products.
A good hit-rate.

I'd taken it on myself to go off and find contracts to keep the company afloat - around $100k/yr. Enough to pay $30k salary and at least one of the directors. This initiative didn't help my stock there.

In 1988, I got to know Ken Thompson reasonably well when he visited Sydney University and taught for a year.

1989
I'd worked too hard, become 'burnt out' left Softway and moved 'to the country', the South Coast of NSW.

I approached the Telecom Product Development Fund with a proposal to build a single TCP/IP based network, using commodity Intel hardware running Unix, to switch all their 'Text Services" [Telex, Teletex, Fax, email, file transfer].

I'd worked for 7 years on exchanges at O.T.C. and knew a bunch about Telco services and digital networking and messaging. This was before Telecom/Telstra had an email service.

I'd run the O.T.C. commercial  e-mail offering, "Minerva", as well as multiple in-house Unix systems using Sendmail, ACSnet/MHSnet and even some uucp.

My proposal included a store-and-forward network and 3 possible variants of fax machines - that would use DNS-style human readable addresses, not just phone numbers.

For ordinary fax machines, this meant having a human-readable cover-sheet - eliminating wrong numbers.

Telstra's comments were:

• We don't know what you're proposing. It's not one thing.
  
• We don't have any one person who can evaluate this.
• Nobody else in the world is doing this.
• Your cost estimates are too low. [commodity hardware vs. 'telco' pricing]
• Your cost/benefit estimates are wrong. [no help on what would be right]
• From the chief of Text Services in Melbourne on consolidating all text switching:
  "Why would we want to do that?"

That was the year that AARNET brought the Internet to Australia.

1992
While at CSIRO, built the first widely available complete Australian daily weather database (software).
Created a browsable index, redefined the file structure, performed full-coverage testing on the dodgy code I'd inherited - and got the product released at the annual ABARE 'Outlook' conference admist much fanfare.

1994
Designed a VPN for linux. Implemented by David B Deaves.
Worked between Canberra and Perth offices over public Internet.

1995/6
Filed, had examined and granted my 'netserver' petty patent.
A domestic router, firewall and internet connection. Also allowed USB devices to be connected.
The petty patent lasted 5 years and was only for Australia.
Unable to find a VC interested in pursuing this.

1999
For Federal Govt. "Business Entry Point" administered the initial ABN registrations.
Forecast peak demand to within 5% and designed the first web 'busy tone' to handle (massively) excessive loads. Designed and specified "Busy Tone" load control/ shedding system. CMGA paper with Performance Dynamics, Dr Gunther.
Final peak load was 25-times the design load, with consistent 5 second response times.
Only part of ATO process to avoid negative press: 600,000 applications processed of 3.3M.
Expected range was 5%-10% of ~1M applications.

2001
Filed an international PCT application for a Power Change-over device [allows servers or a rack of equipment to be switched to another power outlet without disruption.]
Examined and granted in Australia. Others now sell commercial versions.

2005/6
A new take on Internet Security - probably provable secure.
Introduces a replacement for VPN's:  Virtual Air Gapped Networks and Controlled Secure Links.
By adding other elements it is suitable for Military/Government grade security as well as normal commercial operations and domestic/e-commerce use.
Global market for VPN and related devices is currently estimated around US$25Bn.
Replacing and upgrading VPN technology would probably create a market 10-20 times larger.

Patent application filed in Australia on 27-September-2006.
Refiled in 2007, now lapsed.

Historical Storage Prices - Raw Data

These are the raw data I collected from some Australian PC magazines from 2002-2007.



There are problems with the data:

- these are not 'best retail price' - just what I could find advertised

- inconsistent markup and discount schedule. Couldn't find a single advertiser.

- not same-same. RAM was DDR 3200, then PC133

- Disks. Ignored cache size, and SATA.

- all drives assumed to be 7200rpm

- didn't collect data on 2.5" drives and SCSI/fast drives.

RAM

Year	Size	32	64	128	256	512	1024
2002	512			$117	$238	$510
2003	512			$51	$96	$202
2004	512					$149
2005	1024					$168	$189
2006	1024						$135
2007	1024						$159

Flash

Year	Size		32	64	128	256	512	1024	2048	4096	8192	16384
2002	256		$53	$87	$147	$305
2003	512			$63	$109	$187	$350
2004	1024		$39	$49	$75	$129	$200	$400
2005	1024				$40	$60	$90	$158
2006	2048						$55	$95	$175
2007	4096						$25	$45	$69	$99
MSY.COM	$						$15	$15	$25	$45	$92	$177
	$/Gb						30	15	12.5	11.25	11.5	11.0625

Disk

Year	Size		40	60	80	120	160	250	320	400	500	750
2002	120		$205	$275	$375	$375
2003	160		$165	$190	$220
2004	250		$95		$115	$155	$215
2005	320		$80		$90	$129	$135	$206
2006	500				$75		$109	$155
2007	750				$86		$100	$133	$133	$149	$186	$345

DVD - 50 pieces

Year	Size		4.7 Gb
2002			$135 (guess)
2003			$90
2004			$60
2005			$40
2006	8x		$40
2007	16x		$35

RAM

Year	Best $/Gb	Yr/Yr Ratio
2002	936	2.4375
2003	384	1.2886
2004	298	1.5767
2005	189	1.4000
2006	135	0.8491
2007	159
Avg		1.510373357

Flash

Year	Best $/Gb	Yr/Yr Ratio
2002	1176	1.68
2003	700	1.75
2004	400	2.53164557
2005	158	1.805714286
2006	87.5	3.535353535
2007	24.75	2.237288136
Avg		2.260542678

Disk

Year	Best $/Gb	Yr/Yr Ratio
2002	3.125	1.136363636
2003	2.75	2.129032258
2004	1.291666667	1.567556634
2005	0.824	1.329032258
2006	0.62	1.666666667
2007	0.372
Avg		1.565730291

DVD

Year	Best $/Gb	Yr/Yr Ratio
2002	0.574468085	1.5
2003	0.382978723	1.5
2004	0.255319149	1.5
2005	0.170212766	1
2006	0.170212766	1.142857143
2007	0.14893617
Avg		1.328571429

Summary

The Yr/Yr ratios are used for forward projections.

The 'Est Flash' column uses the current 'best price' (MSY) for flash memory and a Yr/Yr ratio of 3.25.

Flash memory is now very much cheaper than RAM - forward projections not done.

Year	RAM $/Gb	Flash $/Gb	Est Flash $/Gb	Disk$/Gb	DVD $/Gb	Max flash	Max Disk
2002	936.00	1176.00	1176.00	3.13	0.57	256	120
2003	384.00	700.00	700.00	2.75	0.38	512	160
2004	298.00	400.00	400.00	1.29	0.26	1024	250
2005	189.00	158.00	158.00	0.82	0.17	1024	320
2006	135.00	87.50	87.50	0.62	0.17	2048	500
2007	159.00	24.75	11.06	0.37	0.15	4096	750
Yr/Yr ratio		2.26	3.25	1.57	1.33
2008		10.95	3.40	0.24	0.11
2009		4.84	1.05	0.15	0.08
2010		2.14	0.32	0.10	0.06
2011		0.95	0.10	0.06	0.05
2012		0.42	0.03	0.04	0.04

Blogging Tools

Google's 'Blogger' is cheap and cheerful and a reasonable way to get started... But it doesn't work well for me in a number of ways.

• 'Safari' on Mac isn't well supported
• The edit window is very small
• I can't compose off-line and upload previous work simply
• My articles aren't stored on my computer making it easy for me to re-use and organise them.
• The dinky little input/edit window isn't resizable
• Checking final layout is painful - flipping in and out of 'preview' mode
• Using Blogger and it's AJAX interface kicks up Safari's memory use - and kills my machine

So I went searching for some tools. Their are Mac clients that look good - but it's pay, pay, pay all the way on Mac :-)

Downloaded 'BloGTK' - it might work in the X-11 environment, but that uses too much memory. I 'only' have 512Mb - which once upon a time would've seemed unlimited.

So I found 'flock'. It's a modified Firefox browser which understands blogging and actually has resizable windows and stores new material on my disk... But it doesn't allow me to import what I've already written. And the publish interface is incredibly annoying if you editing/updating.

And there's a Firefox addon 'Firescribe' (formerly Performancing). It does some stuff reasonably well. Not sure about saving posts locally.

Still searching for tools I can work with ;-)

News-May-11

Microsoft Naysayer bandwagon gets crowded: http://blogs.zdnet.com/BTL/?p=4820

Sarah Friar of Goldman Sachs:
It's over after Vista.
Microsoft needs markets that can move the revenue needle.
Microsoft will be hurt by Linux, Apple and software as a service.
Microsoft can't be nimble enough to compete with Google.
Microsoft's messaging stinks. [what's "Live"?]

"It all seems a bit premature. Sure Google may be sexier–just ask the Microsoft employees that are defecting and burning bridges on the way out–but can you really say a company with $28.8 billion in cash is on the downswing? I can't."
[http://blogs.zdnet.com/microsoft/?p=380]
-> ROE is 35+%. Should be able to repeat that
-> $17B 2006/7 spent on share buy-backs => propping up the share price



Chunks: hidden key to RAID Perf. http://blogs.zdnet.com/storage/?p=130
* Cache
* Striping
* Chunk Size [size of stripe on each drive]

Web DNA - Attributor: http://blogs.zdnet.com/micro-markets/?p=1321
-> Copyright Control, Monetisation On-line

Typical Linux Desktop User
http://blogs.zdnet.com/hardware/?p=397
-> http://www.desktoplinux.com/news/NS9755856281.html

2008 Year of Mobile Linux? http://blogs.zdnet.com/mobile-gadgeteer/?p=359

RedHat takes OLPC knowledge corporate: (one Laptop per child) Global Desktop http://blogs.zdnet.com/BTL/?p=5022

New Rules for Software VC. http://www.sandhill.com/opinion/editorial.php?id=132&page=4

Here is a list of must-haves to “jump start a startup” today and make it a candidate for venture investing.
* Begin with a first-class, seasoned management team

* Find a market undergoing demand growth where some form of disruption is creating a space for a new business

* Buy the components of the business – technology, distribution, customer base - that you can; only build what you have to (which usually does include building innovative technology

The Enterprise Software “Threatdown” http://www.sandhill.com/opinion/editorial.php?id=131 [Top 10 threats]

1. Buyer Ennui
2. Looming Skills Shortage
3. Software Security
4. Usability
5. India’s IT Services Firms
6. Private Equity Plays
7. New ERP Challengers
8. The On-Demand Model
9. Mobile
10. Software Appliances

SandHill: The State of the Software Industry – 2007
http://www.sandhill.com/opinion/editorial_print.php?id=133
Innovation Accelerating as a Top Priority
Internal Funding is Healthy -> revenue growing 9-13%
+ Margins are equally healthy. Megavendors (>$10B Rev) margins 35-40% EBITDA.
+ Margins rest 12-18%
+ software share of the customers’ IT budget will continue to increase
++ 2006: 29.6% expected 35.7% in 2009
External Funding Driven by Private Equity
+ Compound Annual Growth (CAGR) 26% in VC investment
Implications for Software Vendors and Customers
+ Customers: Customers have a paradox in front of them – how can a company that is increasingly spending their budget with the largest software vendors ensure they get access to smaller vendors who are driving innovation?
+ Vendors: Software vendors need to adapt to the bottom-up and center-out diffusion model within this innovation wave.



Sand Hill 30 - The stocks driving the next generation of software and services.
http://www.sandhill.com/finance/sandhill_index.php

Megavendors
Pure-play software vendors whose every move shakes the industry.
HP
IBM
Microsoft
Oracle
SAP

Big Shots
Well-positioned, large vendors in healthy sectors.
Adobe
BEA
CA
EMC
Intuit
Symantec

Specialists
Vendors with deep expertise and a strong market position.
Autodesk
Blackbaud
Business Objects
Cognos
Hyperion Solutions
Informatica
Lawson Software
Sybase
Tibco Software

New Big 5
The most powerful offshore IT services providers in the world.
Cognizant Technology Solutions
Infosys Technologies Ltd.
Satyam Computer Svcs.
Tata Consultancy Services
Wipro Ltd.

New Agers
Next-generation vendors leveraging new technologies and models.
Red Hat
RightNow Technologies
Salesforce.com
Taleo
WebEx Communications

What Price a Life? II

How do we individually and collectively survive and thrive?
What do we need to know from the past to avoid the mistakes of our forefathers?
How will our future depend on what we do in the present?

Reframing what looks like a question about death and mortality into a much deeper, broader question that can surprise and sometimes delight the viewer but always takes unexpected paths. There is enough variety in the question to make informative, provactive media for more than 10 years. Real information on our collective choices and their consequences will help shape the political debate and agenda in a unique way.

Having a Life

In 1901 Australia had a population of 3.8M. In 2006, it was well over 20M.
In 1911, 30% of the population worked in Agriculture, which accounted for 20+% of GDP. Over 40% of the population lived in rural areas. Most families were >4 children.
In 2006 this was around 4% and 3% of GDP. 90+% of the population lived in urban areas. Families are <=2.
ABS - 2001 Yearbook, "A century of population change in Australia".

AIHW life expectancy data:
1900 M:55 F:58
2000 M:77 F:82

In the 1950's the average family, with 4 children, was supported on a single wage or income:
- the average wage in a city
- 40 acres (16ha) in a 'mixed farm' (dairy, pigs, poultry, crops)
- or an author writing "one book a year" [no citation]

In the 2000's, two above average wages are needed to purchase a house, women are deferring children until age 32, 400ha is not economic for broad-acre or grazing enterprises - and authors need 8-10 books 'in print' at one time to make a living.

We are all better off - we must be. But at what price? Why does it take so much more to pay for our lives? Where does the money go...

Taking a Life

The headline meaning, "what does it cost to end a life?" can be examined in many ways.

For example comparing and contrasting the dollar cost per life in warfare.

• the monetary cost of killing a person in War over centuries - the American Civil War, The Boer War, WWI, WWII, Korea, Vietnam, Gulf War I & II.
• What was the cost the USA of bombing Afganistan "back into the Dark Ages"?
  - and what about the cost to the Taliban and the Afhanhi community there and around the world?
• And the low-tech version, How much money did it cost in Mogadisheu to wield machtees?
  - and what was the collectve cost of Ethnic Cleansing to their country and the countries that have provided aid for decades?
  

What is the effect on the global and individual economies of the Arms Suppliers (the USA, UK, France & Germany), the arms users and the Arms trade blackmarket?

But the world doesn't need just another War Documentary, even if it does take a different view.

And their is an individual, personal side to this equation:
What is the time, energy and opportunities passed up for a single man to fight for the US military?

If he does come home in one piece, what are the long-term effects on him, his family, his community?
And for those that increasingly come back as 'damaged goods', what of their lives afterwards?

Saving a Life

Every year hospitals, paramedics and rescue crews save thousands of road accicent victims.

There is a very real dollar cost in saving, or trying to save each of these people. It's part of our culture that we want to offer the same helping hand to everyone. But are we that equitable and elgalitarian? Are there places we quietly just don't serve because the people are somehow 'not as worthy' or 'the costs are too high'. Did the community make those decisions - or have them made for us?

But the victims and their families have to live after 'the accident'. Head injuries are often life-changing. Personalities change - people become violent, abusive and irrational. It's hard for them, and harder for their carers.

How do the paid professionals cope, the families and friends and those that used to depend on them?

And how do we individually and collectively deal with the knowledge that many were preventable?
We are better at not killing people on the roads than China and South Africia. But can we do better?
Commercial Aviation shows what can be done - if there is a will.

But do we want to pay that price? Discipline, rigour and attention to detail. And maybe not everyone automatically has 'the right to drive, to handle a lethal weapon'. How many people are we happy to see die needlessly by our collective inaction?

Politicians won't frame that question. But if they don't, who will?

Supporting us in the lifestyle to which we've become accustomed

We sit on the terrace and sip a nice latte or fine chardonay - but there are consequences for the rest of the world.

All the people who laboured to bring a simple thing together.
And to keep us safe while there and getting there.
We have water to drink, electricity to heat and light, phones to chat upon and buildings that stand up.
But most of all, we are blessed with abundance. We don't toil in the fields or labour in factories. Never before in history have so many had such a fine life. Many things we take for granted - even as a right - were once not even the domain of Kings. Refrigeration, detergents, clean running water, safe food, antibiotics, microsurgery, phones, TV, computers, the 'Net...

These we take for granted... It wasn't always so - and isn't so for the majority of the Earth's population.

And we are borrowing from the future, from our childrens' inheritance, to support this lifestyle.

"Water security" is now Real News. Wasn't it always? Folks in the bush have always been there.
Soon we'll have to consider the consequences of the drought - erosion, loss of soil and unavailability of food and goods.
And the profiteering that will go alongside scarity.

And where do our farmers go when the fertiliser runs out?

Oil is too precious to be burn like trash - look around you and find something that is not composed at least partly of plastics. It is either there because of petro-chemicals used in its production/manufacture or in its transport and use. When Oil runs out, the true cost will not be 'oh no, I can't drive', but 'where's all the consumer products?'.

Conclusion

The concept is to present difficult questions about how we live our lives and the implicit and explicit choices we make - in a sensitive and insightful manner. To ask questions to which there are no simple answers - only choices. Where 'living in the question' is the only right-minded response.

To provoke informed debate. To say things and address issues in areas politicians fear to tread.
And hopefully grab an audience and entrall them. To take them on an unpredictable, exciting journey.

What price a Life?

This is an idea for a probably infinite set of media programs... That could inform, entertain and may provoke (shock, horror!) some thought/introspection.

In 3 or 4 generations we in the Western World have gone from a farming and manufacturing economy and society, to a 'service economy'. Just one generation saw - electricity reticulation, street lighting, radio, automobile, aircraft, the 'production line', industrial scale War [WW I] and the end of domination of European Royalty and Nobels.

We've gone onto create new life forms and manipulating DNA. To knowing most of the building blocks of matter and creating nuclear bombs capable of extinguishing everyone on the planet many times over. And to generations living off social welfare, using drugs and escaping into the 'virtual worlds' of TV, computer games and on the 'Net.

We didn't get here by accident, but we could lose everything in a flash with an accident - or through others intentionally destroying it.
Appreciating what we have and why we have it is essential to preserving our lives.

"What Price a Life?" It's ambiguous and can be taken many, many ways. Such as:

Price

'price' has many dimensions:

• Dollars
• Time
• Energy
• Resources (animal, mineral, vegetable)
• Opportunity Cost - the cost of choosing 'this', not something else
• Environmental
• Human - personal, family, friends, group, community, town, country, ethic grouping
• Human - physical, mental, emotional, spiritual
• Sociological
• Consequences - what are the on-going results of decisions, actions and inaction?

a Life - I

"What price for Taking a LIfe?" With a gun, a knife, .... Intentionally and accidentally. In War, in peace and at home.

Or:

• Saving a Life - hospital, road safety, industrial safety, nuclear
• Extending Lives - good health and nutrition
• Prolonging - last month, 3 months, 12 months
• Living a Life. e.g. a monk, a celebrity, a cleaner, a guard, a doctor, a teacher, a prostitue, a farmer, ...
• Creating a Life - IVF, surrogacy, Gene Therapy...
• Selling/Trading a Life - slave traders, 'black-birding', white slavery, sex slavery, ..
• Preventing/Containing a Life - jail, detention, ...
• Seeking a New Life - immigrants, refugees, ...
• Freedom, Self-Determination, Self-Expression, Creativity
• Justice, Equity, Fairness

a Life - II

"What Price a Life here and now?"
Or:

• Over time
• In this town, community, country
• For this sub-group - aboriginals, Native Red Indians, Inuit
• By religon/religous belief
• By Sex (M/F/other)
• By sub-culture - terrorists, drug-pushers, regime...

a Life - III

"What Price a Human Life?"
Or:

• A wild animal
• A pet
• A domestic animal (food, fibre, fleece, milk, eggs, ...)
• An endangered species
• A not 'cute and cuddly' thing
• Insects, parasites, pests...

Request to Twinings Tea Company: Create more business through the Net.

How to INCREASE your profits and keep old fogeys like me happy...

PLEASE bring back your wonderful variety of leaf tea!

History

Since the early 1970's I've been discovering and drinking your teas. There are other brands - but I like yours :-)

In 1986 at an IT startup company I helped found we collected & drank over 30 varieties of your teas - as well as our own Espresso machine. Wonderful times!

Around 1988 you changed the formulation of "Earl Grey" and the colour of the packaging. Tasted like Burgamont oil from mint not oranges.

And since you've increasingly focused on what can be easily sold in the high-street and supermarkets. Limited range and culling slow-sellers. Migrating to tea-bags.

ALL GOOD THINGS!!
All great, not merely good, business sense.

New Business Model

How can you increase your profits and cater to old fogeys like me???

A. The Net - direct sales with minimum qty.
B. Regular "speciality packs" into supermarkets.

Internet sales

• The people who want to buy your specialist leaf tea are buying a luxury item. A higher price marks extra value. And yes, there would be a price-point sales will decline.
• you already have many national distributors & can supply locally (by mail) to most of the world.
• You get to charge double for smaller batches that you make less often.
• Selling direct means you cut out the distributor/reseller - and double your margin again.
• And you don't have to do anything much different. Every country now has Internet Sales 'Fulfillment' organisations that charge a modest fee and do everything else...
• minimum quantity sales [like 1Kg or $25] means your profits don't get sucked away by transaction fees.

In the Supermarkets

• Look at Aldi and their approach to 'special items'. They make them available for a limited period and then swap out the stock for new specials. You can do a different speciality pack every 3 months for 4 weeks. Savvy buyers will stock up.
• In Australia, well known brands have occasional 'nostalgia' packaging. Tins printed to celebrate an anniversary usually. They aren't real 'collectables' - but a nice promotional vehicle that in today's recycling conscious world don't get trashed for quite some time. They are a long-term in-home promotional display - that you make an upfront profit on!
  [I have a 250g Earl Grey tin bought around 1990 that sits on my counter top & I see everyday.]

Generation X & Y effects

It's true the Baby Boomers [1946-1964] have a lot stronger connection to Ye Olde Teas - and all the ceremony surrounding them. They also have very high purchasing power now, like what they like (are brand loyal) and are generally comforted by links to their 'roots' - especially pleasant/fond memories.

Their children, Gen X, are surprising - they're turning 40 and have taken on many of their parents preferences and patterns. Why do the Beatles and Rolling Stones still sell so well?? Gen X & Y know and share many of their parents preferences.

And you could be tapping that market as well...

Market Testing

Australia is a great place to test :-) Two large supermarket chains with 60%+ coverage, 80%+ internet access/use [home or office] and a bunch of cashed-up baby boomers - and a reliable & cheap parcel post system.
Won't cost you much to market test and establish the vectors/relationships.

I suspect there is a HUGE pent-up demand for some of your now premium product, from a small but loyal band of 'believers'.
Surveys are unreliable - Dollars at the cashpoint are the only survey that matters.

Summary

You can have both the new market of limited choice, fast/easy prep., high turnover + lower margins
AND
Ye Olde World "Slow Food"-style premium-product with high-margins and spectacular range.

The Internet has changed everything. Let it change your business too.

New Blogger Interface

Previously, wrote notes on the Blogger Interface. Was it ATOM? (Probably) These are not meant to be instructions! They are notes on the process of using the API manually New API doco:

http://code.google.com/apis/blogger/gdata.html#Get_Feed_Manual

"To get a feed of a blog's entries, you send the following HTTP request to Blogger" [Everything in XML!]

curl -o blog.1 http://itilopia.blogspot.com/feeds/posts/default

"After you send the GET request, Blogger may return a redirect, depending on various factors. If so, send another GET request to the redirect URL." Curl takes care of this... Posting: Client Login [URL encoding needs to be done in the POSTS]

Then authenticate the user. To do that, send a POST request to the following URL: https://www.google.com/accounts/ClientLogin

From http://code.google.com/apis/accounts/AuthForInstalledApps.html

POST /accounts/ClientLogin HTTP/1.0 Content-type: application/x-www-form-urlencoded Email=johndoe@gmail.com&Passwd=north23AZ&service=blogger&source=Gulp-CalGulp-1.05

---

[CAPTCHA example] POST /accounts/ClientLogin HTTP/1.0 Content-type: application/x-www-form-urlencoded accountType=HOSTED_OR_GOOGLE&Email=johndoe@gmail.com&Passwd=north23AZ&service=blogger& source=Gulp-CalGulp-1.05&logintoken=DQAAAGgA...dkI1LK9&logincaptcha=brinmar

Where

logintoken	(optional) Token representing the specific CAPTCHA challenge. Google supplies this token and the CAPTCHA image URL in a login failed response with the error code "CaptchaRequired".
logincaptcha	(optional) String entered by the user as an answer to a CAPTCHA challenge.

curl's POST doesn't work... This URL has a working example for calendar using wget...

http://denizyuret.blogspot.com/2006/05/google-calendar-how-to-delete-multiple.html

$ wget --no-check-certificate -O blog.2 --post-data 'Email=stevej098@gmail.com&Passwd=xxxxxxx&service=blogger&source=Gulp-CalGulp-1.05' https://www.google.com/accounts/ClientLogin $ grep Auth blog.2 To use the 'Auth' string: wget -q -O- --header='Authorization: GoogleLogin auth=XXX' http://... And this piece from the instructions is cracker-jack! You gotta go look at the

Now send the POST request to the appropriate Blogger URL: http://www.blogger.com/feeds/blogID/posts/default Note: This URL is the same as the URL in the <link rel="service.post"> tag that appears in the <head> section of the human-readable version of the blog.

It also appears in the XML downloaded: <code><entry><id>tag:blogger.com,1999:blog-3110200838296430371.post-4190105953841900395</id> And this is how you push it in :-)

In the body of the new POST request, place the Atom element you created above, using the application/atom+xml content type.

Final command: a=$(grep Auth blog.2) wget -q -O- --header="Authorization: GoogleLogin $a" --header 'content-type: application/atom+xml' --post-file=blog-entry.xml http://www.blogger.com/feeds/3110200838296430371/posts/default

List all Blogs owned by user

Save XML description of all users blogs & tags into file 'blog.4' wget -q -Oblog.4 --header="Authorization: GoogleLogin $a" http://www.blogger.com/feeds/default/blogs neat. Something worked as advertised...

Query a Blog

Save XML of all blog entries made in March, 2007 (in blog.5) wget -O blog.5 'http://itilopia.blogspot.com/feeds/posts/default?updated-min=2007-03-01T00:00:00&updated-max=2007-03-31T23:59:59' Cute quote from doco (I wonder where to find *that* list?):

To query using other standard GData query parameters, just change the URL to use other parameters. (Note that Blogger doesn't support GData category queries or full-text search queries.)

Which might be here:

http://www.google.com/base/api/demo/html/demo.html#query

Deletes and Update/Edits

Too hard... Not clear. no examples of fully generated URI's. Tried to generate a URI for an item/article - failed. here's the suggestions. untested... Note the edit URI's of the events you'd like to delete: wget ... | tidy -xml -wrap 999 | grep edit Delete the event by using its edit URI; this is a two step process, because wget does not redirect correctly: wget -nv -O- --header='Authorization: GoogleLogin auth=XXX' --header='X-HTTP-Method-Override: DELETE' --post-data='' 'URI' Watch the output and note the redirect URI with gsessionid attached, then execute: wget -nv -O- --header='Authorization: GoogleLogin auth=XXX' --header='X-HTTP-Method-Override: DELETE' --post-data='' 'URI?gsessionid=YYY'

Demonacracy - Putting the Demons back into Democracy

In this posting on "Why all Monopolies become Corrupt", I was flying without a parachute - using a browser that didn't support Google's spell-checker. So I banged it out and published - only to get pinged by a friend on this great typo: "Demoncracy". I love the term - it does represent what many small-l liberals of both Left and Right think is happening these days with "machine men" politics feed by Big Money Political Parties. In Australia, we are governed by under the 5% view. With ~10M voters, there are fewer members of the main political parties than your average good sports club. [Described with a little more colour in the last few years in Adelaide, South Australia.] There are fewer than 500,000 - and probably under 100,000 - members of the main political parties. The Parties represent themselves and the views of their members, the rest of us have the option of "take the full package - you'll love it" or not... It's often Hobson's Choice - neither is remotely close to what you might want if you had the choice. And professional politicians (an oxymoron?) consistently rank as more disliked/untrusted than any other profession. Exactly the opposite applies to indepentents elected to Federal Parliment - they are seen are highly principled and ethical. Australians being "very laid back" (what used to be called apathetic) just go along with this state of affairs. We're not lotus eaters nor univolved and uncaring - just wear the wrong clothes to a weekend football match! We're mostly 'comfortable' and just getting on with our lives - Politics is often seen as an a three-yearly annonyance. A view I'm sure the people of Eastern Europe that recently put their lives on the line to have a Democracy don't share... We have bills to pay and places to go - and active lives to lead. Politics can take care of itself - it has done so all this time! Our current Political Party Machine system is a direct result of the statistical work of George Horace Gallup in the 1930's that led to his correct prediction, against all other pundits, of the 1936 US Presidential election by Franklin Roosevelt. For fascinating, first hand accounts of this, see Alistair Cookes' "Letter from America" of Monday, 7 October, 2002 - What the American people want and Monday, 17 July, 2000- Now read on... The system of Polls driving elections, speeches and policy results in very little difference between parties. It looks like "Tweedle Dee and Dumber".

All Monoplies tend to become corrupt

This is not about I.T., but we have to work in the real world :-) It does affect us and our work.

That's a Big Statement: ALL Monoplies tend to become corrupt.
The codasyl is, given enought time.

The reasoning/explantion is very simpe - comes from feedback theory.
For any effective control system you need a controller, input and output sensors and two sets of actuators. These form a feedback system that attempts to maintain the system at a 'set-point'. There's the whole open/closed loop control thing. 'Open' means you do no control. 'Closed' systems are what I'm considering here.

The input and output sensors are needed to feed the model embedded in the controller.
- the output is the state you want near the 'set-point'. You have to know if you are on-target or not.
- input levels/rates/values are needed to drive the model the controller embodies.

The actuators adjust the controllable input - one set to increase, another to set decrease. There may be some inputs to the process that you can't control. Such environmental inputs - like air or water. You can't control their pressure, temperature, contaminants, ...

The key here is that you need two or more counter-opposing forces. If you have only one, then you can only hold a system where it is (if it's very stable) or drive it in one direction. The limiting case is the system fails or "max's out" - it goes to a maximum value and stays there. The stuck throttle problem - one that no driver wants to encounter.

Applying Control System Theory to Organisations

Management is supposed to be the internal control sytem for organisations. But who keeps them honest? Who checks the output/operations of any organisation?

The feedback system here is external to the organisation:
How do the behaviour and attitudes/thinking of organisations get held to acceptable values?

There has to be external one or more 'counter-vailing' forces that can enforce change - as well as transparent or obvious 'outputs' of the organisation.

Demoncratic Governments enshrine these principles with the "Separation of Powers". There are at least 3 opposing forces: the Parliment, the Executive and the Judicary. Within the Parliment, there are often two 'houses' - offering the possiblility of opposing forces. Within the process of Government, there are many 'checks and balances' - various bodies, such as the Audit Office, Way and Means committee and Productivity Commission, that can both observe the outputs of agencies and enforce changes.

Outside of both public and private sectors is the Media. Their role, beside pedalling 'shock and awe', is to make the 'outputs' of organisations transparent and visible/open. Democracies then rely on the sensitivity of politicians to 'the ballot box' to enforce change - eventually. Taking away/limiting 'free speech' by fiat or threat (of police/military action) or removing the effectiveness elections removes any consequences - short of armed rebellion/uprising.

Publicly held companies are required to report their financial results - and woe betide any executive that tries to lie about these. This provides some level of 'transparent outcomes' with regulatory control. The counter-vailing forces are the executives versus the regulators.

In a market with many suppliers, customers vote with their feet - they dump suppliers who's cost, quality or service are poor.
The counter-vailing forces are the supplier management versus the customer choice. The customer is free to withdraw from or got to a particular supplier. The management can manipulate the internals of the company to maximise performance against their goals in the context of market preferences and constraints.

In a business monopoly, customers are not free to choose another supplier. The external feedback system, the lose customers actuator, is removed. A commercial monopoly can do what it likes with impunity - until some natural limiter operates or another effect, such as government intervention, occurs.

But there are very large Public Sector monopolies as well:
- Police
- Military
- (public) Health
- (public) Education, primary, secondary, tertiary
- Local Councils
- 'Secret Squirrel' agencies - like Military Intelligence and other "National Interest Classified" organisations.
- Government bureaucrasies

These organisations are notionally audited ('opposed') by Ombudsmen and Auditors.

But the one essential management output is never reviewed independently - managerial decisions.

All these organisations tend to become overtly corrupt - money or power & influence - or covertly - lassitude, perverse behaviours, institutionalised bullying, harrasment and discrimination, structural inefficiences and ineffectiveness, and completely ignoring or disrepecting those whose service they are in (their ultimate employers) - the public.

For secret organisations, the lack of transparent outputs is even worse. The 'independent' reviewers are impossible to find - only people from within the closed community (they are 'cleared') are allowed to view the evidence and ask questions. A free-floating monoculture very different from the enclosing Society and the organisations commission, can and will develop.

Summary

Monopolies are bad.
Unrestrainted monopolies are worse - eventually becoming a Law unto Themselves - with either over or covert corruption.
Private Sector monopolies are the worst. There are no effective mechanisims to control them.

Effective, frequent independent external reviews of both managerial decisions and 'Corporate Culture' [the behaviour, values, morals and ethics shown, not espoused] are essential to prevent 'free floating monocultures'.

Ideally, three counter-vailing forces are needed, in case any one becomes too strong or one of the 'checkers' becomes too weak.

Then you are left with the problem of: "Who watches the Watchers?"
Us, via the media, transparent/open government and 'free speech'.

Memory Tags and JVM's

Came up with this talking to my mate Stanley over a coffee...

Why do Java Virtual Machines have a hard time finding/locating pointers? They need to do this as a precursor to doing a "Garbage Collection". At CMGA conference I talked to an admin from a bank (St George?) who said SUN talked about these "Stop The World" events. With more memory, they take longer - e.g. 2-5 seconds of clock time. And nothing can happen in the JVM while garbage collection is going on... The events may not be able to be scheduled, and their occurrence will be unpredictable.

One of the good points is that more CPU's get the job done faster.
Each n-byte block of memory is read to see if it couldbe a pointer, i.e. has the value in the range of the heap addresses. Then that block has to be examined to see if it's an object, and does it contain pointers, and so on. They are looking for objects that are no longer used and can be returned to the 'free pool'... It sounds a very hard way to get things done.

The Burroughs B5000 used Memory Tagsand descriptorsto describe to the hardware the contents/use of each 'word'. Descriptors allowed real Virtual Memory around a decade ahead of IBM 360/370.

And memory 'tags' eliminated a whole class of problems. The 'Mark Stack' tag prevented stack manipulation [negative stack offsets weren't allowed either] and accidental corruptions. A whole class of security problem and coding faults were avoided.

So my question: JVM's are in complete control of their environment. They know when an object is created. Why don't they 'tag' each object and pointers to objects? This makes the garbage problem simple. You know the location of all objects and pointers. I guess it could be extended to a 'reference count', but I don't know enough about implementing these things...

Won't it take memory space?
yes - this is a space/time tradeoff.

Because we don't have hardware memory tags, then we have to do it for ourselves...
Like the 'free block list' in some file systems, a bit-map of each "object sized" memory region need be kept. I'm thinking that objects can't be less than 64-bits and they'd be allocated on those boundaries.

And the same for pointers - when an object pointer is allocated/created, set the tag for it.

No idea if this has already been tried... Just seems like A Good Idea :-)

Test Scan

This is a page scanned from my notebook today.
The image should be on the left. The text on the right. (default)

I orginally uploaded the image to another blog and have copied the HTML here.

They are stored on PIXCA I think.
Have seen mention of 300Mb total.

Some limits on size of individual image to upload. in the 50Kb or Mb range.