Yet another mp3 player

I last wrote on this topic at the end of August, exactly five months ago, saying "I have ordered yet another player from Ebay but this has yet to arrive so I can't comment on it". The player is a 4MB Ruizo: it has both internal memory and can accept a memory card. So loading this player simply involved me extracting the SD card from the Sansa Clip and inserting it into this player.

The good points of this player: it seems to run forever without being charged and it has a good sound. More importantly, it continues playing from exactly where it was when it was turned off, so theoretically this is the perfect mp3 player.

Why am I less than enthusiastic? The 'user interface' is extremely badly designed, seeming to me non-intuitive and inconsistent. When scrolling though a list, it seems obvious (to me, at least) to use the up and down arrows, but the interface requires using the left and right arrows. Every menu option seems to use the arrow keys in a different manner. Changing the volume is likely to succeed 50% of the time; the other 50% will do something random. I only discovered the 'continue playing from the same place' by serendipity a few weeks ago.

Another small but important point: the Sansa Clip had, as its name suggests, a clip which allows the player to be clipped on to a piece of clothing or to a belt. The Ruizo has a hole at the top which allows a lanyard (a piece of cord) to be attached; presumably this lanyard is then supposed to go around one's neck allowing the player to be like a medal. This is unsatisfactory. After thinking about this for a while, I realised that I can use a key chain - treat the player as a key, then hang the chain on my belt.

Since then I have ceased to use any of the player's options. I turn the player on - it starts from where it stopped previously. When I finish, I press on the round button in the middle for a long time (a few seconds). If I am lucky, there will appear on the screen 1 2 3; I continue to press on the round button until these figures disappear,  to be replaced by 'Bye bye'. I then turn off the player with the on/off switch. 

These simple operations give me what I need. Anything else is liable to pick a random song, or one which is much earlier in the playlist. As I have over a thousand songs on the card, I haven't got to the end yet, so I don't know whether the list will 'wrap around' and start again from the beginning. I'm not even sure if the songs are being played from the card or from internal memory; at one time I tried to set up a play list, but this seemed to play only songs from internal memory so I copied as many songs as I could from the card to the memory.

If it weren't for these interface problems, this would be an excellent player. A set of instructions came with the player, but these were largely incomprehensible, and as I wrote earlier, key usage is inconsistent. At the moment, this is the best player which I have, but it needs improvement.

Thank heaven for system restore

Yesterday ended with me writing "I posted a question on the program's support forum". Due to the day's frustration, I was pessimistic about the chances of receiving a useful reply. Someone did post an answer, most of which was about updating drivers and similar. But the first sentence was the key: restore the system from a system restore point.

I was pleased to discover that my computer creates a system restore point every morning, so I could restore everything to how it was yesterday at 6am. After rebooting, I started the multi-track program, and yes! There was now audible output through the computer's sound card.

For about ten minutes, I tried recording vocals with the microphone connected directly to the computer's sound card but this was not good. So I then connected the external sound card to a USB port, connected the headphones and microphones ... and was able to make very clear recordings.

I sang the song through once, which seemed to be sufficient. Just at the end of the first verse, my telephone beeped which caused me to record a 'drop in' on the final line. Whilst editing the sound file, I saw that the beep occurred just after my singing, so I was able to silence this without needing the drop in. I then spent an hour or so tuning the lead vocal as well as separating strategic parts so that I could create harmonies.

I'm now listening to the finished song and everything seems to be fine. I'll master a version and listen to that for the next few days.

Frustrating day

Yesterday I surprised myself slightly by writing the final verse to the song that I wrote about a few months ago. Every so often I would tinker with the arrangement slightly, but finding words - or even a topic - seemed to be beyond me. About a month ago, I managed to write almost two verses, but threw them out a day later as they seemed to be repeating lyrically what I had written in an earlier song and they sounded very false. Since then, I have managed to write a verse a week, and yesterday, as I mentioned above, I wrote the final verse.

Today I had plenty of time on my hands, so I thought that I would record the vocals. I setup the microphone on the mike stand, plugged in and recorded a few takes. When I listened to them, the tracks had a terrible audio quality, as if they had been recorded at the bottom of a big hole. No amount of equalisation could save them. So I exchanged the microphone and headphones for the combination phones/mike which I used for several years, and recorded a few more takes. Whilst these sounded better, they were still very boxy. OK: I tried yet another microphone which I have. Slightly better but still bad.

At this stage, it occurred to me that the problem might not be with the microphones but rather with the cheap external sound card which I am using, that allows me to connect the microphone and phones to the computer via a USB connector. Years ago, I used to be able to connect the microphone to my 'music' computer and record with the multi-track recording program which I have. At some stage, recordings made this way started sounding bad, which is when I started to use the external sound card with Audacity to record on a mobile computer. Maybe I could try recording directly again.

At this point, the multi-track software ceased working: I get error messages about the sound card not being able to handle input. I tried different configurations but was unable to hear anything from this program. I should point out that other programs - wave editors and mp3 players - work correctly. Strangely enough, I was even able to use the multi-track program to mix the tracks which I had created, although this was not a good idea as the vocal wasn't synchronised with the music, apart from the poor quality of the sound.

So not only did I waste several hours trying to get a good vocal sound, I also managed to ruin the setup of the program which I need in order to turn the vocals into something which sounds good. Very frustrating.

I posted a question on the program's support forum, but I am not optimistic about getting a useful answer, primarily because I am using a very old version of the program.

Validity styles

Over the past six months, one of the psychologists employed by the OP for whom I consult has been working on developing new models, which explain inter alia the 'validity style' of an examinee's responses. These can vary from 'random' (in which the examinee hits one key repeatedly without reading the questions) to 'honest' and on to 'normal' as a default value. The psychologist sent me a list of rules which enable the program which displays an examinee's results to determine the style. He wrote the rules in pseudo-code:

If (INF == extreme && IR == extreme) then return RANDOM.
If (INF == extreme) then return INFREQUENT.
If (IR == extreme) then return INCONSISTENT.
If (DEF == extreme || GI == extreme || 
    (DEF == borderline && GI == borderline)) then return FAKE_GOOD.
If (DEF == low) then return HONEST.
If (CM == extreme) then return UNCOMMON.
(Else) return UNCLASSIFIED.

I have implemented similar rule mechanisms in the past, but in each case they were always 'and' statements, like the first rule shown above. 'Or' statements seem to be more complicated, and 'short-circuit evaluation' should be used throughout. There was one rule which had both 'and' and 'or' statements which had me literally scratching my head how to implement this.

At one stage, I half seriously considered embedding a Prolog interpreter into the program, as these rules could easily be rephrased as Prolog rules, such as

random:- inf (extreme), it (extreme).
infrequent:- inf (extreme).
inconsistent:- ir (extreme).
fake_good:- def (extreme).
fake_good:- gi (extreme).
fake_good:- def (borderline), gi (borderline).
honest:- def (low).
uncommon:- cm (extreme).
unclassified:- true.

but I quickly backed down from this idea. The above mental exercise reminded me that in Prolog, 'or' statements are handled by having two similar rules. After this insight, it became easier to translate the rules into data stored in SQL tables.

Writing the interface code to store the rules was complicated as I identified three different possibilities: creating a rule with a clause, editing that clause and adding a new clause to an existing rule. Thus for 'or' statements, there would be two (or more) rules whereas an 'and' statement would require one rule with two (or more) clauses. Once I worked this out in my head, the implementation simplified.

Writing the code to evaluate the rules (the equivalent of a Prolog interpreter) took some time, primarily because I confused the short circuit evaluation with the value returned by the evaluation function as to whether a rule was proven. After debugging this code live several times, I finally figured out how it should be written. Testing the code on real data seemed to work, but in almost every case, the result was 'normal', which didn't really test anything.

So I added a 'validity style' field to the 'people' table and wrote an external program based on my code which would evaluate for all people who have been tested since 01/01/17 (about 4500 people) their validity style and store this value in the table. I ran the program then queried the table to find people who did not have a normal style. The result was no one, which I knew was incorrect. I went over the code line by line which seemed to be correct, but then I noticed that one query was not closed. This wouldn't make any difference if only one person is being tested (as in the production code), but for my testing code this was a bug. So I added the 'close' statement, ran the test again, and discovered the many people who have non-normal validity styles.

The next step is to check the complete output for these people in order to see whether the raw data supports the validity style. This isn't my problem.

Under pressure but trying to escape

I see that it's been a few weeks since I last posted something here. That doesn't surprise me very much as nothing particularly interesting has happened so far this month, and I've been under a great deal of pressure (or, quite possibly, created for myself the pressure). I have known for two weeks that there would be two difficult meetings in which I had to participate - one at home (kibbutz) and one at work; as it happened, they were held day after day. The meetings have now passed; I won't say that I came out of them unscathed, but at least they weren't as bad as I might have suspected.

Another weight on my mind has been my doctorate: specifically, a failure to chase after people in order to interview them. As it happens, yesterday I completed an interview with one of my (former) pupils then wrote up the interview. Apart from asking former pupils, I also send 'cold emails' to any Priority using company that orders from my company - I can see from their order form which program they are using. This morning I received an answer from one of these cold emails, that someone is prepared to be interviewed! I must follow this up and interview the person next week. I should point out for my readers that these interviews last for about an hour and are conducted by means of the telephone. I downloaded an app for my phone last week which automatically records telephone calls; this worked well yesterday. In the evening I copied the files (the interview was interrupted a few times) from the telephone to my computer; I then listened to the interview and filled out the written report, stopping the interview whenever necessary. I am also translating and paraphrasing as I write.

I have also been planning this year's holiday which will be in Greece: some time in Athens and some time in Andros, which is a very green island (as opposed to Santorini) which is close to the mainland (the ferry ride is 'only' two hours). After playing around with dates and flights, I was able to book the hotel which we wanted in Andros, but was unable to do so with my first and second choices of hotel in Athens. It seems that there is something important happening on May 22nd, because both hotels were full on that date but had rooms for the previous and following days. Anyway, I booked with a third hotel which looks reasonably good.

And somewhere in the background, I have been listening frequently to the song "We can be together" by Jefferson Airplane. I first heard this song some years ago after having downloaded the 'Volunteers' album but wasn't particularly impressed - the song shares the same opening riff with the song 'Volunteers'.  Somehow recently my attention returned to this song and I've been listening to it closely in the past few days. I especially like the vocal harmonies of the 'chorus' and also appreciate the cognitive dissonance between the singing and the words, especially those of the second 'chorus' ("Up against the wall..."). There is a live version of this song on YouTube from the Dick Cavett show, broadcast the day after Woodstock (August 19, 1969) which is quite infamous, as the song was broadcast without censorship (listen and you'll hear why).  

A very good article about this song can be read here. I listen to the music much more closely to the words and can see that the article mentioned above misses a few musical things. Musically, the live Cavett version is inferior to the studio version which is not too surprising as there are some very interesting rhythmic changes happening throughout the song, which get somewhat mangled when played live. Most of the song has a straight-forward 4/4 beat (and one clip shows drummer Spencer Dryden thumping his drums in a normal rock rhythm) but other parts have a slowed down, possibly 3/4, rhythm. 

An SQL trap and how to get out of it

Today I spent about two hours trying to debug a complicated Priority procedure. After about half an hour, I managed to determine exactly where the problem lay, but I couldn't figure out why there was a problem. As it's cold and wet outside, I couldn't get up and go for a walk, turning the problem over in my head. Having a break for lunch didn't help either.

Here's the problematic code

INSERT INTO GLOB_TMPTREE (FATHER, SON, PIS, ORDI, USER)
SELECT :PART, ORDERITEMS.PART, TEC_ORDIFEATURES.OPTIONCODE,
ORDERITEMS.ORDI, SQL.USER
FROM ORDERITEMS, TEC_ORDIFEATURES, TEC_FEATURES
WHERE ORDERITEMS.ORDI = TEC_ORDIFEATURES.ORDI
AND TEC_ORDIFEATURES.FEATURE = TEC_FEATURES.FEATURE
AND TEC_FEATURES.FEATURECODE = 'PIS'
AND ORDERITEMS.ORD = :ORD
AND ORDERITEMS.TEC_U1 = :U1
AND ORDERITEMS.TEC_U2 = :U2
AND ORDERITEMS.ORDI <> :XLINE;

Leaving aside the specific details of the query, I am selecting data then entering it into a table. I checked (several times) that the actual query was returning data; even so, it was not being inserted into the table. The line before the quoted passage deletes all the lines in the table belonging to the current user so there shouldn't be a problem in inserting the data.

With the benefit of 100% hindsight, I can now say: when an insert fails, check the table's primary key. In this case, the primary key was composed of father, son and pis; it so happened that another user had tried to run the procedure but for some reason, the procedure didn't complete and certainly didn't clear up after itself. There were lines in the table for this other user - and it turned out that the primary key values for these lines were exactly the same as the ones I was trying to insert.

When I finally got to this stage, I deleted all the lines in the table (this is only a temporary table with a few lines at most); I then ran the procedure, and unsurprisingly it did exactly what it was supposed to do. I then added the 'user' field to the primary key which should prevent the same error from happening again.

So: when an insert fails, check the table's primary key! Maybe next time I'll remember this.

Love is all you need (film)

I had recorded some time ago a film whose unpromising blurb was something like 'Romantic comedy in which Pierce Brosnan attends his son's wedding in Italy'. As our Yes Max video recorder is due to be replaced today, I thought it prudent to actually watch the film before it gets lost. The first few minutes were disconcerting: a woman had gone to see an oncologist ... and they were speaking in some foreign language which I originally took be Swedish but turned out to be Danish. Then there was a scene with Brosnan as the CEO of a vegetable/fruit company ... not very promising. I fast forwarded about 15 minutes to a 'meet cute' between the above woman and Brosnan which was taken straight out of 'Cold Feet': the woman backs into Brosnan's car in an airport car park. This reminded me of the exchange between Adam and Rachel in CF: "Who was your driving instructor, Stevie Wonder?" "At least I had a driving instructor".

Once they calm down a little, it turns out that the woman is flying to Italy to attend the wedding of her daughter; strangely enough, Brosnan is flying to Italy to attend the wedding of his son. Of course, it is her daughter marrying his son. Cut to a local Italian airport and then to a drive along countryside which looks extremely familiar: they were driving on the road between Sorrento and Amalfi! From this point on, the charm of this area took over the film, which improved immensely. There were many romantic shots of the gulf of Naples, both early morning and evening, which placed Brosnan's villas in the hills above Sorrento, in the Sant' Agata area.

Apart from general scenic shots, there were a few places which I recognised immediately. There were a few shots of Marina Grande, which is right next to where we stayed in Sorrento the first time. When the ladies went "into town to buy clothes", there was a shot which started with the bell tower then ran down the lane where the 100m pizza was served and into a junction where we ate Gnocchi alla Sorrentina. The wedding took place in the cloisters of San Francesco (pictured on the left). It should be noted that all the locations in Sorrento looked cleaner and less run down than they did when we were there. The film was released in 2012, meaning that it was probably filmed a year earlier, whereas we didn't go to Sorrento until 2014, so the intervening years may have contributed to removing the sparkle from the locations.

There was one thing which nagged at me: wedding guests are seen arriving and departing from Marina Grande by boat. Apart from the fact that most boats leave from Marina Piccolo, why would one travel by boat from Sorrento to Naples (airport)? Obviously, the scenic factor plays a part, but Brosnan was shown arriving by car. Only when watching the credits did this become clear: they all arrived at Salerno airport, not Naples. This explains how come Brosnan travelled by car along the Amalfi coast and why the guests travelled by boat.

In the end, I found the film much more enjoyable than the opening minutes suggested.

Amos Oz (1939-2018)

I have just learned that esteemed Israeli novelist Amos Oz has died at the age of 79 from cancer.

I first became aware of Oz around 1975, when I read the English translation of his (then) most famous book, 'My Michael'. This was a story set in Jerusalem just after the creation of the state of Israel. Probably most of the allusions passed me by, but I remember going past certain buildings and thinking to myself that these were mentioned in the book. I read another of his books in translation a year or two later ('Elsewhere perhaps') which was set in a fictional kibbutz, one which was quite unlike the kibbutzim which I had already begun to know and love.

I emigrated to Israel in 1978, to a kibbutz called Mishmar David, whose closest neighbour is kibbutz Hulda, where Amos Oz was living. I remember that during the first weekend after emigrating, a group of us went to Hulda in order to use their swimming pool (we didn't have one at the time in Mishmar David). If I remember the story correctly, after we had left the pool, Amos Oz arrived with Jane Fonda.

A few years later, in an attempt to improve relations between the two kibbutzim (there were no arguments between the two ...), probably at Sukkot (when it is traditional to entertain guests), we went to Hulda and were hosted by several families. My wife was talking to the family that entertained us about their house, and our hostess said something like "I'll show you something else". She went outside, knocked on the door of the next apartment and was bidden to enter with us. "Thank you, Amos", she said. Only afterwards did my wife realise that we had been in the house of Amos Oz.

An apocryphal story which was making the rounds at this time had the treasurer of Hulda calling upon Oz and telling him that he was the most successful branch of the kibbutz (in monetary terms); the treasurer wanted to improve the income of this branch by allowing a few pensioners to work with Oz and so increase the work force.... Maybe this story loses something in translation.

A few years later, Oz and his family moved to the desert town of Arad in order to ease his son's asthma, and so our paths ceased to cross.

The last book of his which I bought was a non-fiction book about the Israeli elections in 1982, when Oz visited several cities, towns and settlements.

First official picture

24 hours and 180 degrees

Yesterday, shall we say, was not a very good day. In the morning, I travelled to Tel Aviv and got soaked to the skin. I don't like rain very much, but once I was wet, it didn't bother me any more as I couldn't get any wetter than I already was. I dried off slightly whilst in the train, but the rain at the other end of the journey ensured that I arrived home wet - I went straight into the shower, taking off my wet clothing and hanging it to dry. To add insult to injury, I tried to get some cash from the ATM at our local bank branch; the machine swallowed the card!

Overnight 70 mm (2.8inches) of rain fell on the kibbutz, which is a very large amount! About 11% of our annual rainfall. I was about to take the dog for a walk at about 4pm when there was a break in the rain, but by the time I had put on my raincoat (fortunately I have two, for the first got soaked in the morning) and put the dog on the lead, the rain had started again. We managed a walk later on in the evening.

This morning, a telephone call woke us at 5:40am; it was our daughter telling us that it was time for her to go to the hospital in order to give birth to her second daughter. At 10:30am, she sent us a message telling us that she had given birth to a 3.20kg baby! Our other grand-daughter has been with us for most of the day and will stay with us until tomorrow evening when presumably her father will come back to the kibbutz (he will stay overnight with his family in Jerusalem).

I had a nagging feeling that there was something important about 7 December, but it took me a while to figure out what it was. Aside from it being the sixth candle of Chanuka, it is also the 70th birthday of our kibbutz! So we'll always remember this date! Since the formal announcement via the kibbutz website an hour ago, our phones have been beeping every few minutes with congratulations from our friends.

There's no formal picture of our new grand-daughter, but we do have a picture of us lighting the candles.

And to add icing to the cake, I was able to get my debit card back from the bank and was able to withdraw cash.

My world has changed 180 degrees in the last 24 hours!

Executing a program and waiting for it to finish

At the beginning of the month, I wrote "The next stage in the process is to convert the xlsx file (after completion) to a tab delimited file by means of the increasingly valuable program which I wrote with Delphi 10. This turns out to be much harder than expected - a topic for another blog entry."

Here is that second blog entry. In the past, I have used the ShellExecute procedure in Windows in order to execute a program in the context of another program; here is some sample code (taken from a 'program launcher'):

chdir (extractfiledir (path));
cmd:= ini.ReadString (key, 'Cmdline', '') + ' ' + pdata + #0;
ShellExecute (handle, 'open', pchar (path), pchar (cmd), nil, 
              sw_show)

Whilst this procedure does what it is supposed to do, the executing program has no way of knowing when the child program finishes. Sometimes this is not a problem, but sometimes it is. In the context of converting an xlsx file to tab delimited, it is very important to know when the child program has completed.

There is a second procedure in Windows called ShellExecuteEx which both executes the child program and also waits for it to finish. I am sure that I used this procedure once in a program, but I couldn't find the code and so had to rewrite it from scratch. Unfortunately, some of the examples which I looked at were misleading, so it took me some time to figure out exactly what had to be done. Here is the code which executes the converter program:

var
 exInfo: TShellExecuteInfo;
 exitcode: DWORD;
 prog: string;

begin
 With exInfo Do
  begin
   cbSize:= Sizeof (exInfo); 
   fMask:= SEE_MASK_NOCLOSEPROCESS or SEE_MASK_FLAG_DDEWAIT;
   Wnd:= aHandle;
   lpVerb:= 'open';
   lpFile:= pchar (extractfilename (prog));
   prog:= extractfiledir (prog);
   if pos (' ', prog) > 0 then prog:= '"' + prog + '"';
   lpDirectory:= pchar (prog);
   lpParameters:= pchar ('"' + f1 + '" "' + f2 + '" ' + flags);
   nShow:= SW_HIDE;
  end;

 if ShellExecuteEx (@exInfo) then
  begin
   while GetExitCodeProcess (exinfo.hProcess, exitcode)
         and (exitcode = STILL_ACTIVE) do Sleep (500);
   CloseHandle (exinfo.hProcess );
  end;

I discovered that if the program to be executed has a space in its name (or more likely, in the directory name), then one has to put quotation marks around the name. Similarly, the parameters have to be protected by quotation marks. Since having written this code, it works very well in two programs of mine which execute the converter program.

This morning I had an epiphany about the above code: in a different program suite, there is a loader program which receives as input data from an examinee along with a list of exams (i.e. programs) that the examinee has to take. Each exam has to signal the loader program when it has been completed in order to cause the next exam to be executed. The loader program uses the ShellExecute method, which solves half of the problem; the second half of the problem (notification) is achieved by means of a complicated system of inter-program messages. Sometimes there are bugs with this method. It occurred to me this morning that I can drop all the message creating and receiving code: the launcher can use ShellExecuteEx in order to execute each exam, and of course, the launcher will receive a reliable message when the child program has completed. The child program doesn't even have to know that it is being run by the launcher, which simplifies a few things.

Intermediate thesis accepted!!

Following is the text of a letter which I received from the university today.

Dear No’am

I trust you are well.

The Doctoral Review Committee considered your intermediate thesis at their meeting this morning. I am delighted to inform you that they consider it to be of a very good standard and are happy for you to proceed to the main study.

You will soon receive detailed feedback from the reviewers via your supervisor. You are encouraged to discuss the feedback with your supervisor and address any issues to your supervisor’s satisfaction before progressing to the next stage.

Obviously it's a good idea to have undertaken a failed doctoral project before embarking on a successful one. The first time around, it took me eight months and one and a half attempts to get my research proposal accepted; second time, it took about three months, fighting my mentor almost all the way, to have the proposal accepted at its first meeting. The first time, it took about two years and two attempts to have the intermediate thesis accepted; second time, it took exactly one year and one attempt!

Of course, it's not such a good idea to have a failed project as it ate time and money which could have been put to better purposes. On the other hand, the experience gained from failing allowed me to know in advance what I needed to know in order to succeed. 

So what lies ahead? First, I have to see the detailed feedback - I suspect that the paragraph about feedback in the letter is boilerplate. Assuming that there is nothing serious there, the next step is to start the interviewing process. At the same time, I have to discuss how results will be presented in the thesis. It would have been much clearer how to present results in a thesis based on statistics which prove or disprove hypotheses.

E Dorian

Throughout October, I was toying with a chord sequence which I developed, trying to turn it into a song. I think that the fact that I was toying or dabbling means that I wasn't very happy with it. About a week ago, I sat at the piano and played the sequence once again, trying to find a tune which would work; after knocking around for a bit, I discovered that I was playing a song written by someone else (I don't recall now which song, not that it matters) and that the rhythm had changed from 4/4 to 6/8. I started to sequence this variation, but didn't get very far.

After a few days cogitation, I decided to start afresh, keeping only an instrumental portion from the song. I decided arbitrarily that the song would use the E Dorian scale (E F# G A B C# D), and unusually for me, all the chords used (at least, in the verse) are based on this scale: Em F#m G A Bm and D. I quickly developed a 15 bar sequence for the verse and even built a complete arrangement. 

As someone noted on the Music Practice and Theory stack exchange site, this scale doesn't really have the concept of perfect cadences (e.g. B -> Em); the normal cadence for the Dorian mode is D -> Em. Thinking about this, I decided to reverse the chords in the last three bars of the verse, from Em D A to A D Em, as this would have a better 'cadential' feel. This meant that I had to change several bars in the arrangement. Sitting down once more at the piano in order to develop a tune, I realised that I needed a 16th bar to the verse; back to the sequencer in order to add this bar to every verse, and then restructure the drum loops. The tune may be a bit too high for me to sing comfortably: it ranges from D above middle C to C# an octave higher. Lowering the key by two or three semitones would bring it into my comfort zone, although I have reached that high C# without help in the past.

That's all behind me now. The song - or rather, the arrangement - is now complete, at least until I have some more ideas. All that's left to do is write the words. I haven't given this much thought as I was waiting for the melody to settle down. The current idea is to call the song 'Wide awake', where the lyrics are something like 'Wide awake at five past one, the daily grind has just begun; wide awake at ten past two, ......'. I don't know whether those words even fit the new rhythm and I'm unsure as to whether this is a topic which will maintain an entire song.

Improving a solution

I thought that I had written about a problem which I had with loading external files containing Hebrew into Priority; the Hebrew text wasn't being reversed when read by Priority (or rather, it was being reversed but not unreversed). One day, I realised that I could write a program which would reverse the Hebrew in advance (this would have the Hebrew left to right). Once read into Priority, the left to right Hebrew would then be corrected to right to left, which is the way it should be.

The original data flow was:

1. Copy the external file to a special directory, changing its name (this is so the interface program in Priority can find its input)
2. Read this file into a holding area within Priority, very basic structure
3. Move the data from the holding area into another holding area with better defined structure
4. Convert the data in the holding area to the customer order format.

The problem with the Hebrew was in the first stage: the basic holding area had reversed Hebrew strings. The original program - which stopped working - corrected the data in this holding area, between stages 2 and 3. As I am not able to write a program which accesses data within Priority tables, this option was denied to me, so I decided to attack the first stage: my program ('RevHebrew') altered the original external file prior to its being read into Priority.

At some stage, I realised that I only need create a backup file if RevHebrew actually reversed anything in the file; this cut the number of backup files by at least 50%.

A few days ago, it occurred to me that I could simplify matters if I used RevHebrew at a later stage. I wrote a new version of the program which didn't save its original file with a new extension (this required removing exactly one line from the original program) and set this to work on the file which has been copied to the special directory. 

The benefits of this are two-fold: there is no 'original backup' file, and the file which is backed up by the Priority program has the Hebrew in its correct form, not in reversed form. Thus if we ever need to input the file again (which I sometimes do in order to check where a problem occurred with a file's contents), there won't be a problem with the Hebrew.

The programming house who were supposed to check the original program and write a replacement had the cheek to send me a bill for work done. I returned this, noting that their 'solution' is not a solution and that the assigned programmer obviously did not understand the problem, judging by his solution. This was only yesterday, so of course I have yet to receive any response from them.

Improving Bill of Materials data in Priority

I have often wanted in Priority that the lines in the Bills of Material (BOM) would show when a given line was added. Looking at the screen which displays this data yesterday, I discovered that there is indeed such a field and that there is a screen trigger which fills that field (and also stores the name of whoever added the line). So why were all our BOMs lacking this data?

I don't know whether the company for which I work is exceptional in this, but I estimate that 99.99% of our BOMs are created by copying previously existing parts. Obviously, changes are made in the BOMs themselves, but again about 99.99% of these are made by a configuration program. It seems as though the copying program and the configurator, both of which are external C programs, directly insert parts into the BOM without using the screen interface. So these lines lack the date they were entered, and by whom.

This is how the COPYPART procedure works: first, the user is asked for the catalogue number of the part which is to be copied. Then an empty screen is presented, in which the user writes the catalogue number(s) of the new part (s) - one can copy an existing part to a new part, or to several parts. Then the copying begins. My code takes advantage of the screen in which the new part numbers are written: these numbers are stored in a table called NPART.

My code runs at the very end of the COPYPART procedure: it loops over all the parts in the NPART table created by the current user and inserts a tuple into the 'extended BOM table', storing the user name and date. PARTARC is the name of the table which stores the BOM data.

DECLARE D1 CURSOR FOR
SELECT PART.PART
FROM NPART, PART
WHERE NPART.PARTNAME = PART.PARTNAME
AND NPART.USER = SQL.USER
AND PART.TYPE = 'P';
OPEN D1;
GOTO 300 WHERE :RETVAL <= 0;  
LABEL 100;
FETCH D1 INTO :NPART;
GOTO 200 WHERE :RETVAL <= 0;
DECLARE D2 CURSOR FOR
SELECT SON, SONACT, ACT, RVFROMDATE
FROM PARTARC
WHERE PART = :NPART;
OPEN D2;
LOOP 100 WHERE :RETVAL <= 0;
LABEL 110;
FETCH D2 INTO :SON, :SONACT, :ACT, :RVDATE;
GOTO 120 WHERE :RETVAL <= 0;
/* This is effectively PARTARC/BUF5 */
INSERT INTO PARTARCA (PART, SON, SONACT, ACT, RVFROMDATE)
VALUES (:NPART, :SON, :SONACT, :ACT, :RVDATE);
UPDATE PARTARCA
SET USER = SQL.USER, UDATE = SQL.DATE
WHERE PART = :NPART
AND SON = :SON
AND SONACT = :SONACT
AND ACT = :ACT
AND RVFROMDATE = :RVDATE;
LOOP 110;
LABEL 120;
CLOSE D2;
LOOP 100;
LABEL 200;
CLOSE D1;
LABEL 300;

The only strange part of this is code which was adapted from the existing screen trigger (PARTARC/BUF5): first an insert is performed into PARTARCA and then the same table is updated. As I understand it, this code first inserts a new record; the insertion could fail if there were no existing record - if someone were editing an existing BOM line. The update naturally updates; this is now guaranteed to work. If there were only an update without an insert, then new lines would not be inserted. Inserting without updating would fail when editing. So this slightly odd approach has been used.

I suppose I could simply have written an insert for my code, as at the time when it executes, there is guaranteed to be no existing line. I've kept it like it is in order not to depart too far from the original.

Intermediate thesis submitted!

Since I last wrote about my thesis, there have been great improvements. I utilised the Succot holiday to improve the research methodology. After this was approved by my advisor, I commenced the pilot study. As I wrote in the thesis:

The pilot study was conducted with a manager responsible inter alia for Priority development, testing the proposed approach and methodology. The objectives of the pilot study were:

• to verify that the questions proposed in section 4.2 are appropriate
• to discover whether there are additional questions which need to be asked
• to determine how much time is needed for an interview
• to determine how much time is needed to analyse the interview
• to determine the suitability of the general structure of the interview.

Originally I reported on the pilot study without any defined structure, but after finding a recent thesis from my department which was accepted, I adopted the structure of the report as appeared in that thesis. I sent this version off to my advisor a few days ago; yesterday evening I received his approval, and so this morning I submitted the thesis to the research committee.

The committee will be sitting towards the end of the month. As with the research proposal, the submission is sent two weeks prior to a meeting of the research committee; two members read the document, make notes and present their findings to the committee which then decides whether to accept or reject the document.

Unlike my first research project, where I adopted the strategy of submitting the first time a version that I think is good but probably not good enough, I have high confidence in this version. As my supervisor wrote yesterday, I think that you and I have spent enough time reviewing this and it now needs some 'external' eyes on it. I think you've really created a document, easy to follow and understand... which is the absolute core skill of a good researcher.

So there's going to be a hold on reporting DBA activity until the end of the month. Ironically, I received an email this morning asking for my quarterly progress report. As my plans for the next quarter depend entirely on the research committee's decision, I think that I will delay sending in the report until the committee's meeting.

Protecting cells in Excel files

Yesterday the OP and I were discussing time sheets: some of her psychologists prefer to fill an Excel worksheet with their hours worked. Looking at this (and bearing in mind recent work of mine in Priority), I said that I could write a program which would input the data from the worksheet into the appropriate table within the OP's management program. Of course, I noted, we would need to have a standard file format, so then I found myself agreeing to write a program which would output such a file.

The first stages in writing the 'calendar' program were very simple; there was a certain amount of 'fun' involving saving the file in xlsx format, but this was swiftly overcome. I realised that I would have to protect the format of the file - I don't want people changing the structure. In order to do this, I would have to protect the cells in two columns (date and day name) whilst leaving the third column (hours worked) editable. I've never done this with an Excel file, let alone with automation, so first I had to learn how to protect the cells, then how to do this within the framework of a Delphi program.

The necessary code appears below:

// allow users to edit only the hours
 sheet.Protection.AllowEditRanges.Add (Title:= 'range1',
                                       range:= sheet.columns['C:C']);
 sheet.protect;
 sheet.Range['C2'].Select;

 // save as xlsx
 XLApp.Workbooks[1].saveas (filename:= copy (s, 1, length (s) - 3) 
                            + 'xlsx',
                            FileFormat:= xlOpenXMLWorkbook);

The final line is required as the Excel file is created by loading a csv file, thus a regular 'save' would save the file as csv, not xlsx.

The next stage in the process is to convert the xlsx file (after completion) to a tab delimited file by means of the increasingly valuable program which I wrote with Delphi 10. This turns out to be much harder than expected - a topic for another blog entry.

Losing an old friend, gaining a new one

Today my mobile computer for work will be replaced. I thought that I had written about receiving it several years ago, but I only noted its predecessor from slightly over five years ago. There were several problems with the Dell that I received then, mainly that there were almost no sockets. I had the computer replaced with a Toshiba Portege Z30T-A which I have liked very much.

True, I've had problems with the jittery mouse, and the network adapter socket sometimes causes problems, but this computer fits neatly into my backpack, it has plenty of memory and has a very fast disk. So what happened?

Last week (Monday), I travelled to Karmiel on the train. I used the computer for about fifteen minutes at the beginning of the journey with power supplied by the battery. After I had been working at Karmiel for a few hours, I noticed that the battery was empty, even though the computer was connected to the mains. In other words, it should have been charged to full and stayed that way. I disconnected the mains, and as expected, the computer turned itself off. I called our technical support (normally they call me about Priority problems); the technician wanted me to remove the battery from the computer, but this computer doesn't allow easy access to its gizzards, so we had to leave things as they were.

The next day, I looked at the computer again: still the battery was empty despite being connected to the mains. Technical support said that they were ordering me a new computer. On Wednesday out of habit, I took out the mobile computer in order to connect to one of the companies for whom I consult - and lo and behold! The battery started charging. I informed technical support, but they said that the new computer had already been ordered, and anyway this computer is 'old' - after five years!

As it happens, I worked early this morning for about an hour with the battery with absolutely no problems. But progress is progress, and so in a few hours time I will be receiving a new computer. Then will start the awkward process of transferring as much as possible; I keep very few data files on this computer so that won't be a problem, but it's the programs (and their associated definitions) that I'm worried about.

---- to be continued on the new computer ---

So here I am on my new Dell Vostro computer, running Windows 10. I spent some time yesterday getting the computer set up the way that I want it. Earlier this morning I imported all my bookmarks along with VPN settings, so almost everything is ready. I had to look up my gmail password; I haven't yet got access to my home router (!) and there are AnyDesk settings to be imported (?) or reset. Otherwise ....

First rain

The weather forecasts have been predicting a rainstorm which is due to hit on Thursday morning and evening, then lightening up. In reality, Thursday morning was overcast, but by lunchtime the clouds had disappeared and it was quite hot. I went to bed early as I have had a very tough week and my eyes were hurting.

At about 10:30pm, I was wakened by a very strange sound, as if someone was moving wooden tiles around on the small balcony outside our bedroom window. Then the rain started. The balcony has a roof consisting of bamboo and plastic which does not let the rain through - but makes a sound when rain falls on it. And what a sound!

I have on my mobile phone an application which I originally loaded in order to measure how much noise a worker makes, close to my cubicle. He is a big man with a loud voice who speaks Russian, which I find extremely distracting. Normally his voice is in the 45-55dB range, but it has passed the 60dB mark and maybe hit 70bB.

The noise from the rain last night reached 70dB, which the application describes as 'busy traffic, vacuum cleaner', so of course it was impossible to sleep at this stage. After at least an hour, the rain eventually weakened, making it was possible to sleep again.

There were puddles everywhere this morning, but clear skies. The sun is shining now and probably causing all the water to evaporate, so by this afternoon there will be few signs of the heavy rain. I haven't seen yet the report of how much rain there was*. In Israel, we don't have the various types of rain that Britain does - drizzle, a polite rain, medium rain or heavy rain. We just have heavy rain, which isn't too good for the ground, as it quickly becomes saturated with water and can't absorb any more. This means that a fair amount of last night's rain will find its way to the sea without being absorbed into the soil. A shame.

* About 36mm, which is about 5% of the annual rainfall.

Conditional choose-field trigger (2)

Almost exactly one year ago, I wrote a convoluted blog entry about choose-field triggers in Priority. It might be that I was wasting my time as there is a documented solution for what I was trying to achieve. From chapter 5 of the Priority SDK:

You can also write a specific CHOOSE-FIELD or SEARCH-FIELD for the procedure. Your trigger can contain references to any input value specified by the user within the same [emphasis mine] procedure step. For instance, if the procedure step contains an input parameter called CST, its value will be stored in the :PROGPAR.CST variable. This is useful, for example, if a given procedure step contains an input column for a Sales Rep and another input column for a Customer, and you want the Choose list for the latter column to display only those customers that are associated with the specified sales rep.

I emphasize the word 'same'. Looking again at the code which I presented in the blog from last year, the 'input value specified by the user' comes from a previous step in the procedure and not the same step.