Sequencing "Lost"

Over the past few weeks, I have been working on sequencing the song "Lost" by Van der Graaf Generator, which appeared on their album 'H to He' in 1970 (either their second or third album, depending on how one counts). This is one of my all-time favourite songs, so much so that I was moved to record a version on cassette played by myself and two friends in 1975 (it's probably not very good). It's about time that a digital version (with controlled vocals and full instrumentation) should be created. There are only two problems with this song: it's very long and there are multiple time signature changes.

The structure of the song can be said to be thus: verse type A (VA), VA, verse type B (VB), verse type C, VA, VB, VB. I suppose that this is somewhat similar to the verse|verse|chorus|middle 8|verse|chorus|chorus structure which appears frequently, but the B verse is definitely not a chorus, and the 'middle bit' can hardly be construed as a 'middle 8'. Not only that, but the A verse is almost a song in itself, being very long and constructed of several sections.

Just to complicate matters, most of the A verses could be considered to be in 3/4 time, but towards the end, the rhythm moves into 4/4. The B verses are in plain 4/4, as is the 'middle bit'. Whilst the sequencing program which I use can handle multiple time signatures, Reason cannot; I solved this problem by having a 4/4 time signature for the entire piece, having the A verses sequenced in 12/8, written as 4/4.

My version is about 255 bars in length and plays for about 12:45 minutes (depending on the fade), running at 80bpm. The original lasts about the same time, but is faster; it contains two instrumental interludes which I elided.

I sequenced the opening verses sometime in 2009; I remember that it took me a long time to achieve something which sounded reasonable - the opening 12/8 section had to be played in a 'dignified' manner and not 'swung'. For reasons which I cannot recall, I left the work in progress to be completed at a later date. I resumed work on the song about three weeks ago, in which I added all the missing parts and then worked on making a complete and coherent arrangement. As it happens, I made very few changes to the first part - an organ riff was added to the opening verse which later transferred to what was originally a flute part. The flute transmogrified into a soprano saxophone at a very late stage. I found some good drum loops which help the final B verses immensely

I now face the awkward prospect of having to sing this thing! I don't expect any problems with the A verses nor with the middle bit, but pitching has been a problem with the B verses.

Stitching the song together has not been easy; I recall that Peter Hammill once related to me how the 'H to He' album was recorded on 16-track, thus enabling them to record songs in pieces, changing instrumentation and ambience. For a long while, I didn't think that this song was part of their live set, but I have at least one complete live performance in my collection and there is also a partial video, which used to be on YouTube.

What am I going to do with the song once it is finished? I can't upload it to SoundClick as I don't have publishing rights for the song and have no intention of seeking them. I'll probably upload it to a file sharing service which I sometime use then inform the PH mailing list about it.

More spooks

I wrote seven weeks ago about watching this prize-winning television series. Then I was in the middle of the fourth series ... little did I know what awaited me. I've just seen the third episode of series 8 and I have to admit to being shocked. Somehow, previous cast deaths had left me somewhat apathetic but today's episode left me speechless.

A few weeks ago I found the tv tropes homepage, a site which describes and notates all the various tropes which are used in television and film. As far as Spooks is concerned, [it was] a show where Anyone Can Die: Most of the time when you finally grasp who the main characters of the story are, you can expect that these characters will survive through the end of the story. Well, this is not that trope.

If earlier series of Spooks hinted that someone was going to disappear by adding a new actor, the past few episodes have seen a total demolition of the staff with no replacements arriving in advance. Thus there is no warning that someone is about to die.....

I have managed to get my wife addicted to the series as well, although she didn't see the early series and hasn't had much time lately to watch. A few weeks ago, when I was in Tel Aviv, I knew that she intended to catch up on a few episodes, including the series 7 opener. She telephoned me to say that she was crying after the death of ...; according to the Spooks forum, this was not an uncommon event. I had purposely not warned her of what was going to happen. Probably by the time she gets to this episode, I will be somewhere else and will have forgotten what happens.

Refactoring

Unfortunately I spent a few days at home last week, mainly in bed; I had a mild case of flu. Fortunately, I didn't run a high temperature, but I was very weak and my head felt like cotton wool for most of the time. But that's all over now....

After my head returned to normal service but before my body regained its strength, I spent some time refactoring the code in the OP's management suite program. As this suite has been a work in progress for more than two years, it's not surprising that certain functionality has been repeated which can now be safely factored out. It turns out that a function to calculate how many days there are in a month appeared in three different units (and at least once I ignored leap years); this was easy to fix.

About 14 months ago, I wrote about sorting data in grids and included some very simple code. When I was looking at the management suite the other day, I realised that several units contained this kind of code,

with qTest do
  begin
   open;
   addindex ('idx0', 'num', [], '', '', 0);
   addindex ('idx1', 'num', [ixDescending], '', '', 0);
   addindex ('idx2', 'extra', [], '', '', 0);
   addindex ('idx3', 'extra', [ixDescending], '', '', 0);
   strings:= tstringlist.create;
   getindexnames (strings);
     strings.free;
   close;
  end;

After some cogitation, I realised that I could make a double saving: I could write a routine which automatically created two indices (one ascending and one descending) per field in the table, and of course I could use this routine in every module. If I remember correctly, about eight modules used this generic code; of course, now I can add it to more modules without occurring any development cost. As a side effect, the size of the executable has shrunk.

As I point out to the OP, centralising code (or refactoring) means (amongst other things) that there is only a single point of failure for each function.

Here is the new code:

Procedure BuildIndices (cds: TClientDataSet);
var
 i, j: integer;
 alist: tstrings;

begin
 with cds do
  begin
   open;
   for i:= 0 to FieldCount - 1 do
    begin
     j:= i * 2;
     addindex ('idx' + inttostr (j), fieldlist.strings[i], [], '', '', 0);
     addindex ('idx' + inttostr (j+1), fieldlist.strings[i], [ixDescending], '', '', 0);
    end;
   alist:= tstringlist.create;
   getindexnames (alist);
   alist.free;
   close;
  end;
end;

Solving the unflushed inventory problem

Just over a year ago, I wrote at length about unflushed inventory. In the year that has passed since then, I have been fighting a mainly unsuccessful battle in order to raise public awareness of the problem. Suddenly, a solution appeared about a month ago.

I was talking with our company's ERP consultant about one of the problems inherent with bills of material (BOM): the BOM tells us how much it costs to make a part and whilst also providing the raw input into the manufacturing process. There have been times when I have needed to divorce these two functions. Even though we have discussed this before, the consultant hadn't provided any kind of solution, but this time he mentioned, off the cuff, that if I mark a certain flag in the parts table, then the part will not be sent to the production floor as a result of a production order.

There are parts whose BOM contains virtual parts such as royalties. We need the BOM to contain the royalty because this contributes a certain amount to the cost of the final product. On the other hand, negative inventory of this 'part' builds up every month which only serves to confuse the inventory system. Once I marked this royalty part with the special flag, it carried out its costing function without interfering with manufacturing. 

Such a small change can produce large gains.

Thinking about this a few weeks later,  I realised that this flag held the key to solving the unflushed inventory problem. To recap: ERP's work order (based on the BOM) says to use a certain amount of raw material (wood), but via the use of an external optimiser, the workers are able to produce the same amount of finished product with less than the dictated amount of raw material. This causes problems with stock keeping and creates what Priority calls unflushed inventory. 

For a while it seemed as if Priority was ignorant of the output from the optimiser, but then I realised that there was a 'warehouse transfer order' entered into Priority which transferred the required amount of wood planks from the raw materials store to the factory floor - this order is the optimiser's output. After some more cogitation, I realised that I could utilise this order: I could convert it into the list of parts which had been removed from the factory floor, consumed by the production process. In other words: the optimiser would cause X amount of planks to be sent to the factory floor, then my program which would remove those X planks as if they had been consumed normally. As the plank parts had been marked with the special flag in Priority, the fact that the default BOMs required Y planks became totally irrelevant.

The required program took a few iterations to write correctly, but after a few days it was working perfectly (it is now run automatically every evening). Every day I check the inventory of the shop floor in order to ensure that no wood planks are listed; if there are, then I correct the faulty definition of the plank in the computer in the knowledge that the plank will be removed from inventory the next time that my program runs.

Thus the unflushed inventory problem has been solved ... at least, as far as wood is concerned. There are still far too many parts to be found in the unflushed inventory, but as most of those parts seem to be screws or similarly small and insignificant parts, I can't get anyone interested in the problem.

Maslow's hammer beats again

The design and engineering departments at work have started to use the SolidWorks (SW) program; it is intended that this program will replace AutoCAD in a few years. Presumably because SW creates more files than AutoCAD, the person running the project is considering using a Product Data Management (PDM) system which integrates with SW (good idea). Unfortunately, the PDM costs a lot of money and the powers that be wish to delay the purchase of a PDM until they see that SW is integrated properly into our work practices (another good idea).

What to do in the interim? The same project manager sent off an email a few weeks ago saying that "a jointly managed Excel file saved on a network drive would maintain the SW file list". I almost had an apoplexy when I read this! I immediately sent back an email stating that Excel files might be fine as long as they are personal, but multi-user Excel files suffer from a multitude of problems, mainly due to file locking. They are also unsophisticated. What we need (I wrote) is to appropriate the database functions of Priority in order to maintain such a list; automatically we will get such advantages as proper multi-user handling, numerators, indices, auditing and text handling (an optional text screen in which users can write comments). 

We had a meeting about this on Sunday, attended by several grey Russians who tend to the inflexible. At least one was horrified by the suggestion that we maintain this database in Priority; he calmed down somewhat when I explained to him that this 'database' would be maintained in a corner of the program and would neither impinge on the rest of the program nor would require the user to be conversant with the rest of the program (although one of the huge benefits of Priority and other ERP programs is that once one has learnt how to manipulate one screen, that person can then manipulate all the other screens in the program with little additional effort, otherwise known as leveraging). Presumably this gentleman is fearful of Priority; I wonder how he is managing to learn SW. It took me about an hour to set up the database tables, screens and triggers necessary to implement what was agreed.

This discussion demonstrated a point which is very close to my heart: unsophisticated people will use a given tool, not because it is the best tool for the job but because it is familiar to them. In other words, this is a case of Maslow's hammer: when the only tool you have is a hammer, every problem looks like a nail. In this case, the hammer is of course Excel.

This is the sort of behaviour which I intend to examine once I start my DBA.

Gateway

I've been traveling frequently in the past few weeks, during which I've been rereading Frederik Pohl's Gateway novels. I very much enjoy these books, as much for the hard science (cosmology) as for the humour (eg "When I was a child, there was a 128 page book sold called "Things we known about the Heechee. All the pages in the book were blank"). 

Today, I came across this statement on a discussion list:

... some things that look like sci-fi are really fantasy. The latter Star Wars movies (episodes 1-3) are like this: their focus on midi-chlorians is no different from a wizard with magic dust. Frederik Pohl's Gateway novels are similar: the blind use of alien artifacts is no different from Bilbo Baggins finding that the ring makes him invisible. This is why the setting for so many fantasy stories is a decaying, once-great society: in ages past the people understood their creation, but the knowledge has been lost.

I beg to differ as regarding Gateway: it's true that the humans initially use the Heechee artifacts without understanding how they work, but they constantly try (during the first two books) to understand how those artifacts do work. Of course, once they find out how the artifacts work, new and surprising uses are found for those artifacts, uses which the Heechee never dreamed of. So this is fantasy?

I own a copy of Pohl's autobiography, "The way the future was", which contains all manner of interesting material. I paraphrase Pohl in writing that every experience is grist to the writer's mill: there's a paragraph about Pohl learning to play the guitar, which seems to be a dead end until one reads the first Gateway book in which Robin Broadhead thinks about an upcoming guitar lesson and how one makes the transition between D7 and C.

A change is gonna come

I saw most of a film called 'Talk to me' on television the other night, an adapted true story of Washington D.C. radio personality Ralph "Petey" Greene, an ex-con who became a popular talk show host and community activist in the 1960s. I wasn't too sure at the beginning whether this was supposed to be a comedy or a drama, and towards the end I got fed up with the film (which is why I didn't see it all). 

For my money, the best and most moving part was a scene which took place in 1968, when Greene had to announce the death by assassination of Dr Martin Luther King Jr. Greene was almost crying when he delivered the news, and the people reacted by setting fire to parts of Washington. It was Greene's job to calm the people down. 

As he finished his midnight - 2am shift, he played a final record, which started off with syrupy strings ("It's Nat King Cole", I thought), then moved into a vocal which sounded as it were sung by Rod Stewart (in other words, Rod Stewart styled his vocals on this singer): "It's been a long time coming, but a change is gonna come". Prophetic words sung beautifully by Sam Cooke.

I have always wondered what the attraction was for young white British youths in the mid 60s for black blues and soul music, and hearing this sad and dramatic song, I finally realised what it was all about. Those post -war youths were looking for direction and found it in this song. True, the arrangement sounds saccharine to my ears and it could easily be improved, but there is no doubting the strength of the song's words and of its delivery.

A Change Is Gonna Come

(Sam Cooke)

I was born by the river in a little tent

And just like the river, I've been running ever since

It's been a long time coming

But I know a change is gonna come

 

It's been too hard living, but I'm afraid to die

I don't know what's up there beyond the sky

It's been a long time coming

But I know a change is gonna come

 

I go to the movie, and I go downtown

Somebody keep telling me "Don't hang around"

It's been a long time coming

But I know a change is gonna come

 

Then I go to my brother and I say, "Brother, help me please"

But he winds up knocking me back down on my knees

 

There've been times that I've thought I couldn't last for long

But now I think I'm able to carry on

It's been a long time coming

But I know a change is gonna come

House with no door

I wrote the other day about receiving the live cd boxset of Peter Hammill. Last night I had the opportunity of listening to one of the discs closely - I chose the disc which I have subtitled "Peter Hammill murders songs by VdGG".

First of all, I doff my hat to someone who is prepared to even try and play these complicated songs on his own. Some of them come off quite well whereas some don't. Hammill has never been known as a particularly good guitarist (he admits as much himself), so the songs which he plays on guitar start with a handicap which he never manages to beat. Some of these guitar songs - especially 'Masks' and 'When she comes' - sound like a 16 year old playing them on guitar. To be modest, I think that I would have played them better than Hammill when I was 16 - except for the fact that the songs had yet to be written. On the other hand, I would never have attempted to play the apocalyptic "After the flood", which actually comes out reasonably well (of course, there's no twelve tone dirge passages in this solo version).

Hammill plays piano much better than he plays guitar, so the piano songs should theoretically sound better, even approaching their original versions. Again, I admire the chutzpa of Hammill to play 'Scorched earth' on his own; whilst certain parts bear almost no resemblance to the original, the majority of the song shows a high accuracy (including the metric irregularities).

The one song which I want to write about most is a song which has been close to my heart since its release in 1971: "House with no door". I even performed this a few times in public around the time of its release (when I was indeed 16 years old) so I have a deep knowledge of it. Hammill  played piano on the recording so he too should be more than familiar with the chord sequence. 

I note that the song is credited to Hammill/Jackson, but I have never been able to ascertain what David Jackson's credit is for. Did he write all the music? Maybe the credit was for his multiple flute/sax solo?

Anyway, in the key of C, the chords to the verse are

C G Dm6 C F Em Bb F

C G Dm6 C F Em Bb Fm

The chords show a beautiful movement;  the change from Em to Bb creates a very strong cross- relation and is most distinctive - to my ears, it makes the verse. Similarly, the F at the end of the first line (there is a double plagal cadence here, Bb -> F -> C) is mirrored and saddened by the Fm in the same place in the second line: another brilliant chord change.

Unfortunately, Hammill almost ignores this sequence: I'm not sure what chord he played after the Dm6,  but the Bb chord is replaced by Dm  (a weak if possible replacement) and the F is replaced by G, which totally ruins the harmony as well as wrecking the double plagal cadence. To make matters worse, the Fm in the second line is also replaced by G! 

Why sabotage one's own song deliberately?

As it happens, I read an interview with Hammill from 2005 the other day which was very illuminating. It goes with great depth into subjects such as Hammill's guitars, his guitar style, his effects pedals, his keyboards, etc - subjects which interest me greatly. For some reason, Hammill comes across as slightly inarticulate regarding these subjects, and it's quite difficult to understand what he really means. He disparages his guitar playing (quite rightly) but also claims that he knows nothing of chords, possibly disingenuously. I find this last statement very hard to believe; maybe he doesn't know about diminished fifths and sharpened ninths, but he should know chords and standard chord sequences. 

Maybe this lack of knowledge about chords is what caused him to murder the lovely piano ballad, 'House'?

A day in the life

I spent yesterday working in Tel Aviv. After a moderately successful day, I left the offices at 4pm, walked to the nearby train station (arrived 16:15) and took a connecting train from the branch stop (16:20) to the main Tel Aviv station (arriving 16:30). At 16:45, the train to Bet Shemesh (and thence to Jerusalem) pulled into the station; it is due to leave at 16:55.

There was a slight delay in leaving the main station, an unusual occurrence. After a few minutes, the train pulled into the next station then started on its way to the third Tel Aviv station. The train never got there: it stopped midway between two stations ... and stayed that way for about two hours!

The information that we passengers received was minimal, possibly because the railway workers trying to fix the problem didn't know what the exact problem was. We were told that it was an electrical fault and that it should be fixed in a few minutes; after a few minutes, we were told that it would take a bit longer ... and a bit longer ... and eventually they gave up giving us updates. Of course, we couldn't get off the train because we were between stations, and as modern trains don't seem to have any windows which can be opened, the air got hot and stale. 

Eventually, the powers that be sent a replacement train which connected up to the far end of my train; the passengers walked through seemingly endless carriages until we all had a seat (and air!), and then the new train set off backwards, to the previous station. Here we all disembarked, at about 19:10. Everyone was worried that we would have to wait for the next scheduled train to Bet Shemesh, due to leave at 20:00, but fortunately a special train was laid on which left at 19:30, meaning that I arrived home at about 20:25. 

All the passengers will get a refund (the office in Bet Shemesh had already photocopied multiple copies of the required form) but of course this won't return the two unpleasant hours that we suffered. It's not clear what kind of refund will be received by those who hold a monthly ticket.

Fortunately, no one panicked nor started shouting loudly. Such behaviour has a tendency to spread quickly and is to no one's advantage. I wouldn't say that anyone was pleased by what happened, but at least we all got through it with a minimum of discomfort. I had my Kindle with me, so I could carry on reading without too much problem (except when the lights went out a few times).

An untypical Thursday

The day started without eating: I was due to have a blood test, in order to establish a base level for cholesterol in my blood. For the next few months, I'll drink a bottle of Danacol every day, and then when I next have my blood tested, we'll be able to see how effective the Danacol actually is. The health system has joined the Internet revolution, so I was able to see the results that same evening, instead of having to wait for as much as a week.

• Cholesterol - 188 mg/dL
• Triglycerides - 135 mg/dL
• Cholesterol HDL - 44 mg/dL
• Cholesterol LDL (calculated) - 117 mg/dL
• Cholesterol/HDL ratio - 4.27

I get the feeling that the dietary changes which I've made in the past few months (ground flax seed and walnuts) have already had an effect on my blood cholesterol level as the above results are better than the last set from six months ago:

• Cholesterol - 205 mg/dL
• Triglycerides - 149 mg/dL
• Cholesterol HDL - 39 mg/dL
• Cholesterol LDL (calculated) - 137 mg/dL
• Cholesterol/HDL ratio - 5.26

Apparently the LDL is calculated by the Freidewald method, where LDL = total cholesterol - HDL - (triglycerides / 5), when the concentrations are expressed as mg/dL. Thus LDL = 205 - 39 - (149/5) = 136. Using Thursday's example, LDL = 188 - 44 - (135/5) = 117. Right on!

During the day, I implemented a procedure for our ERP program at work which corrects for the usage of 'continuous goods'. Items within a product's bill of materials (BOM) can either be discrete or continuous; an example of the first kind of item is a cupboard handle, whereas an example of the second kind is a sheet of wood. ERP handles discrete items very well but has problems with continuous items, especially as there is normally some form of optimisation applied to the quantities needed. In theory, the BOM will say that X square metres of wood are needed whereas in practice, only X - 0.4 square metres were actually used. This optimisation changes from day to day, making reporting more complicated. I thought about this problem once again the other day and this time I came up with a practical plan for handling it. I wrote the procedure (which worked 99% correctly the first time I tried it) and used it on two test items. If the test works as I predict, then I will widen the procedure's scope to handle more items, which will improve our reporting capabilities.

When I came home, I had two packages awaiting me. The first was the book 'Sandy Denny: Reflections on her music', by Philip Ward. Most of the book's contents were not new to me as they had previously appeared on Philip's website, but since the book's publication, those contents have been removed from the site. I was chuffed to notice that my name appears amongst the people named in the book's acknowledgements: whilst I know that I had helped Philip here and there, I wasn't expecting this.

Since buying my Kindle, I've hardly bought any paper books, so it's always a pleasure to order from the Book Depository and receive a book within a few days at a cheaper price than Amazon (whilst Amazon may sell the book for slightly less, they charge an arm and a leg for postage; the Book Depository does not charge for postage which means a big saving for me).

The second package was the seven cd package from Peter Hammill, entitled 'Pno Gtr Vox Box'. To quote the man himself, "[T]his represents a version of the guitar only, piano only shows with which I began my residence in Tokyo in 2010, with performances taken from the Japanese shows and also from the UK shows in the same year. A double CD is, I think, a decent artefact to leave out there as a (semi-) permanent exemplar of Performance. But while compiling the discs from all the available material it came to me that a specific, bigger, thing could also come out of the various shows. Hence, as the first Fie! release of 2012, we have “Pno, Gtr, Vox Box”: a seven cd box set of live songs from the same sources.
 

This one’s going to be strictly limited in edition. Although the Boxes are not individually numbered, I’ve taken the decision that the run will be restricted to the 2,000 which have been manufactured."

My copy came with a signature in green ink on the back of the box; after asking on the PH discussion group, I have come to the conclusion that PH signed this himself. It's not unique, though, as several other have signed copies too. One has to be careful as the ink runs should it get wet!

I haven't had much of a chance to listen to it yet. I've never been a great fan of live Peter Hammill, especially when he is playing the guitar (and one of the discs is entitled "What if there were no piano?), but the little which I have heard seems reasonable. There is even one disc devoted to solo performances of Van der Graaf Generator songs - hearing Hammill play 'Scorched earth' with all its time changes is fascinating!

I wish every Thursday were as good as this one was!