Friday, March 23, 2007

New Blogger Interface

Previously, wrote notes on the Blogger Interface. Was it ATOM? (Probably) These are not meant to be instructions! They are notes on the process of using the API manually New API doco:
http://code.google.com/apis/blogger/gdata.html#Get_Feed_Manual
"To get a feed of a blog's entries, you send the following HTTP request to Blogger" [Everything in XML!]
curl -o blog.1 http://itilopia.blogspot.com/feeds/posts/default
"After you send the GET request, Blogger may return a redirect, depending on various factors. If so, send another GET request to the redirect URL." Curl takes care of this... Posting: Client Login [URL encoding needs to be done in the POSTS]
Then authenticate the user. To do that, send a POST request to the following URL: https://www.google.com/accounts/ClientLogin
From http://code.google.com/apis/accounts/AuthForInstalledApps.html
POST /accounts/ClientLogin HTTP/1.0 Content-type: application/x-www-form-urlencoded Email=johndoe@gmail.com&Passwd=north23AZ&service=blogger&source=Gulp-CalGulp-1.05
[CAPTCHA example] POST /accounts/ClientLogin HTTP/1.0 Content-type: application/x-www-form-urlencoded accountType=HOSTED_OR_GOOGLE&Email=johndoe@gmail.com&Passwd=north23AZ&service=blogger& source=Gulp-CalGulp-1.05&logintoken=DQAAAGgA...dkI1LK9&logincaptcha=brinmar
Where
logintoken(optional) Token representing the specific CAPTCHA challenge. Google supplies this token and the CAPTCHA image URL in a login failed response with the error code "CaptchaRequired".
logincaptcha(optional) String entered by the user as an answer to a CAPTCHA challenge.
curl's POST doesn't work... This URL has a working example for calendar using wget...
http://denizyuret.blogspot.com/2006/05/google-calendar-how-to-delete-multiple.html
$ wget --no-check-certificate -O blog.2 --post-data 'Email=stevej098@gmail.com&Passwd=xxxxxxx&service=blogger&source=Gulp-CalGulp-1.05' https://www.google.com/accounts/ClientLogin $ grep Auth blog.2 To use the 'Auth' string: wget -q -O- --header='Authorization: GoogleLogin auth=XXX' http://... And this piece from the instructions is cracker-jack! You gotta go look at the
Now send the POST request to the appropriate Blogger URL: http://www.blogger.com/feeds/blogID/posts/default Note: This URL is the same as the URL in the <link rel="service.post"> tag that appears in the <head> section of the human-readable version of the blog.
It also appears in the XML downloaded: <code><entry><id>tag:blogger.com,1999:blog-3110200838296430371.post-4190105953841900395</id> And this is how you push it in :-)
In the body of the new POST request, place the Atom element you created above, using the application/atom+xml content type.
Final command: a=$(grep Auth blog.2) wget -q -O- --header="Authorization: GoogleLogin $a" --header 'content-type: application/atom+xml' --post-file=blog-entry.xml http://www.blogger.com/feeds/3110200838296430371/posts/default

List all Blogs owned by user

Save XML description of all users blogs & tags into file 'blog.4' wget -q -Oblog.4 --header="Authorization: GoogleLogin $a" http://www.blogger.com/feeds/default/blogs neat. Something worked as advertised...

Query a Blog

Save XML of all blog entries made in March, 2007 (in blog.5) wget -O blog.5 'http://itilopia.blogspot.com/feeds/posts/default?updated-min=2007-03-01T00:00:00&updated-max=2007-03-31T23:59:59' Cute quote from doco (I wonder where to find *that* list?):
To query using other standard GData query parameters, just change the URL to use other parameters. (Note that Blogger doesn't support GData category queries or full-text search queries.)
Which might be here:
http://www.google.com/base/api/demo/html/demo.html#query

Deletes and Update/Edits

Too hard... Not clear. no examples of fully generated URI's. Tried to generate a URI for an item/article - failed. here's the suggestions. untested... Note the edit URI's of the events you'd like to delete: wget ... | tidy -xml -wrap 999 | grep edit Delete the event by using its edit URI; this is a two step process, because wget does not redirect correctly: wget -nv -O- --header='Authorization: GoogleLogin auth=XXX' --header='X-HTTP-Method-Override: DELETE' --post-data='' 'URI' Watch the output and note the redirect URI with gsessionid attached, then execute: wget -nv -O- --header='Authorization: GoogleLogin auth=XXX' --header='X-HTTP-Method-Override: DELETE' --post-data='' 'URI?gsessionid=YYY'

Tuesday, March 20, 2007

Demonacracy - Putting the Demons back into Democracy

In this posting on "Why all Monopolies become Corrupt", I was flying without a parachute - using a browser that didn't support Google's spell-checker. So I banged it out and published - only to get pinged by a friend on this great typo: "Demoncracy". I love the term - it does represent what many small-l liberals of both Left and Right think is happening these days with "machine men" politics feed by Big Money Political Parties. In Australia, we are governed by under the 5% view. With ~10M voters, there are fewer members of the main political parties than your average good sports club. [Described with a little more colour in the last few years in Adelaide, South Australia.] There are fewer than 500,000 - and probably under 100,000 - members of the main political parties. The Parties represent themselves and the views of their members, the rest of us have the option of "take the full package - you'll love it" or not... It's often Hobson's Choice - neither is remotely close to what you might want if you had the choice. And professional politicians (an oxymoron?) consistently rank as more disliked/untrusted than any other profession. Exactly the opposite applies to indepentents elected to Federal Parliment - they are seen are highly principled and ethical. Australians being "very laid back" (what used to be called apathetic) just go along with this state of affairs. We're not lotus eaters nor univolved and uncaring - just wear the wrong clothes to a weekend football match! We're mostly 'comfortable' and just getting on with our lives - Politics is often seen as an a three-yearly annonyance. A view I'm sure the people of Eastern Europe that recently put their lives on the line to have a Democracy don't share... We have bills to pay and places to go - and active lives to lead. Politics can take care of itself - it has done so all this time! Our current Political Party Machine system is a direct result of the statistical work of George Horace Gallup in the 1930's that led to his correct prediction, against all other pundits, of the 1936 US Presidential election by Franklin Roosevelt. For fascinating, first hand accounts of this, see Alistair Cookes' "Letter from America" of Monday, 7 October, 2002 - What the American people want and Monday, 17 July, 2000- Now read on... The system of Polls driving elections, speeches and policy results in very little difference between parties. It looks like "Tweedle Dee and Dumber".

All Monoplies tend to become corrupt

This is not about I.T., but we have to work in the real world :-) It does affect us and our work.

That's a Big Statement: ALL Monoplies tend to become corrupt.
The codasyl is, given enought time.

The reasoning/explantion is very simpe - comes from feedback theory.
For any effective control system you need a controller, input and output sensors and two sets of actuators. These form a feedback system that attempts to maintain the system at a 'set-point'. There's the whole open/closed loop control thing. 'Open' means you do no control. 'Closed' systems are what I'm considering here.

The input and output sensors are needed to feed the model embedded in the controller.
- the output is the state you want near the 'set-point'. You have to know if you are on-target or not.
- input levels/rates/values are needed to drive the model the controller embodies.

The actuators adjust the controllable input - one set to increase, another to set decrease. There may be some inputs to the process that you can't control. Such environmental inputs - like air or water. You can't control their pressure, temperature, contaminants, ...

The key here is that you need two or more counter-opposing forces. If you have only one, then you can only hold a system where it is (if it's very stable) or drive it in one direction. The limiting case is the system fails or "max's out" - it goes to a maximum value and stays there. The stuck throttle problem - one that no driver wants to encounter.

Applying Control System Theory to Organisations


Management is supposed to be the internal control sytem for organisations. But who keeps them honest? Who checks the output/operations of any organisation?

The feedback system here is external to the organisation:
How do the behaviour and attitudes/thinking of organisations get held to acceptable values?

There has to be external one or more 'counter-vailing' forces that can enforce change - as well as transparent or obvious 'outputs' of the organisation.

Demoncratic Governments enshrine these principles with the "Separation of Powers". There are at least 3 opposing forces: the Parliment, the Executive and the Judicary. Within the Parliment, there are often two 'houses' - offering the possiblility of opposing forces. Within the process of Government, there are many 'checks and balances' - various bodies, such as the Audit Office, Way and Means committee and Productivity Commission, that can both observe the outputs of agencies and enforce changes.

Outside of both public and private sectors is the Media. Their role, beside pedalling 'shock and awe', is to make the 'outputs' of organisations transparent and visible/open. Democracies then rely on the sensitivity of politicians to 'the ballot box' to enforce change - eventually. Taking away/limiting 'free speech' by fiat or threat (of police/military action) or removing the effectiveness elections removes any consequences - short of armed rebellion/uprising.

Publicly held companies are required to report their financial results - and woe betide any executive that tries to lie about these. This provides some level of 'transparent outcomes' with regulatory control. The counter-vailing forces are the executives versus the regulators.

In a market with many suppliers, customers vote with their feet - they dump suppliers who's cost, quality or service are poor.
The counter-vailing forces are the supplier management versus the customer choice. The customer is free to withdraw from or got to a particular supplier. The management can manipulate the internals of the company to maximise performance against their goals in the context of market preferences and constraints.

In a business monopoly, customers are not free to choose another supplier. The external feedback system, the lose customers actuator, is removed. A commercial monopoly can do what it likes with impunity - until some natural limiter operates or another effect, such as government intervention, occurs.

But there are very large Public Sector monopolies as well:
- Police
- Military
- (public) Health
- (public) Education, primary, secondary, tertiary
- Local Councils
- 'Secret Squirrel' agencies - like Military Intelligence and other "National Interest Classified" organisations.
- Government bureaucrasies

These organisations are notionally audited ('opposed') by Ombudsmen and Auditors.

But the one essential management output is never reviewed independently - managerial decisions.

All these organisations tend to become overtly corrupt - money or power & influence - or covertly - lassitude, perverse behaviours, institutionalised bullying, harrasment and discrimination, structural inefficiences and ineffectiveness, and completely ignoring or disrepecting those whose service they are in (their ultimate employers) - the public.

For secret organisations, the lack of transparent outputs is even worse. The 'independent' reviewers are impossible to find - only people from within the closed community (they are 'cleared') are allowed to view the evidence and ask questions. A free-floating monoculture very different from the enclosing Society and the organisations commission, can and will develop.

Summary


Monopolies are bad.
Unrestrainted monopolies are worse - eventually becoming a Law unto Themselves - with either over or covert corruption.
Private Sector monopolies are the worst. There are no effective mechanisims to control them.

Effective, frequent independent external reviews of both managerial decisions and 'Corporate Culture' [the behaviour, values, morals and ethics shown, not espoused] are essential to prevent 'free floating monocultures'.

Ideally, three counter-vailing forces are needed, in case any one becomes too strong or one of the 'checkers' becomes too weak.

Then you are left with the problem of: "Who watches the Watchers?"
Us, via the media, transparent/open government and 'free speech'.

Friday, March 16, 2007

Memory Tags and JVM's

Came up with this talking to my mate Stanley over a coffee...

Why do Java Virtual Machines have a hard time finding/locating pointers? They need to do this as a precursor to doing a "Garbage Collection". At CMGA conference I talked to an admin from a bank (St George?) who said SUN talked about these "Stop The World" events. With more memory, they take longer - e.g. 2-5 seconds of clock time. And nothing can happen in the JVM while garbage collection is going on... The events may not be able to be scheduled, and their occurrence will be unpredictable.

One of the good points is that more CPU's get the job done faster.
Each n-byte block of memory is read to see if it couldbe a pointer, i.e. has the value in the range of the heap addresses. Then that block has to be examined to see if it's an object, and does it contain pointers, and so on. They are looking for objects that are no longer used and can be returned to the 'free pool'... It sounds a very hard way to get things done.

The Burroughs B5000 used Memory Tagsand descriptorsto describe to the hardware the contents/use of each 'word'. Descriptors allowed real Virtual Memory around a decade ahead of IBM 360/370.

And memory 'tags' eliminated a whole class of problems. The 'Mark Stack' tag prevented stack manipulation [negative stack offsets weren't allowed either] and accidental corruptions. A whole class of security problem and coding faults were avoided.

So my question: JVM's are in complete control of their environment. They know when an object is created. Why don't they 'tag' each object and pointers to objects? This makes the garbage problem simple. You know the location of all objects and pointers. I guess it could be extended to a 'reference count', but I don't know enough about implementing these things...

Won't it take memory space?
yes - this is a space/time tradeoff.

Because we don't have hardware memory tags, then we have to do it for ourselves...
Like the 'free block list' in some file systems, a bit-map of each "object sized" memory region need be kept. I'm thinking that objects can't be less than 64-bits and they'd be allocated on those boundaries.

And the same for pointers - when an object pointer is allocated/created, set the tag for it.

No idea if this has already been tried... Just seems like A Good Idea :-)

Thursday, March 01, 2007

Test Scan



This is a page scanned from my notebook today.
The image should be on the left. The text on the right. (default)

I orginally uploaded the image to another blog and have copied the HTML here.

They are stored on PIXCA I think.
Have seen mention of 300Mb total.

Some limits on size of individual image to upload. in the 50Kb or Mb range.