Bedroom lighting almost finished

A small update with some results on the bedroom floor lighting project.

The small red dot in the center of the image above is where the hardware is (the boxed Arduino) with the PIR to the left. It’s hard to make a picture in almost complete darkness – keeping the picture sharp but also getting a result with colors that match with what my own eyes see is not that easy.. But the picture above resembles what I see when I get out of bed during the night well enough – and it’s exactly what I wanted! The LED strips (1.5 meters on both sides of the bed) light up the floor well enough to spot any obstacles, but it doesn’t light up the whole room. And the 15 seconds turn out to be the right amount of time to light up the floor; and the LDR  takes care of unnecessary lighting during daytime; so in one word, the only conclusion can be: perfect! 

One small glitch though, which points out the difference between a clean testing environment and usage in real life: some days ago, when I had to change the alarm clock, I switched on the bedside lamp so I could see the buttons on the alarm clock. This triggered the LED strips to go on!

This means that the switching of the 230V lamp resulted in a ‘spike’ on the PIR sensor input on the Arduino… hmm. I think I’ll have to redo some wiring and use shielded cables instead of the unshielded & untwisted ones I’m using right now. My bad…

Perhaps another solution would be to watch the incoming PIR signal on the Arduino more closely and detect whether it’s a short ‘spike’ or not and decide not to turn on the LEDs on a short ‘high’; but that just doesn’t feel right – cables first!

Jenkins, unsigned integers and VB.NET

Fotunately I had my hair cut short recently, cause otherwise I’d have pulled them all out yesterday!

For a project I’m working on, I need to use the so-called “Jenkins’s one-at-a-time” hash. The algorithm looks like this, in C code:

uint32_t jenkins_one_at_a_time_hash(char *key, size_t len)
{
  uint32_t hash, i;
  for(hash = i = 0; i < len; ++i)
  {
    hash += key[i];
    hash += (hash << 10);
    hash ^= (hash >> 6);
  }
  hash += (hash << 3);
  hash ^= (hash >> 11);
  hash += (hash << 15);
  return hash;
}

No big deal. Right? Until you have to rewrite this to VB.Net. Here’s what I started with, and when things got weird:

Public Function Jenkins_one_at_a_time_hash(ByVal data() As Byte) As UInt32
 
  Dim hash As UInt32 = 0
 
  For Each b As Byte In data
   hash = hash + b
   hash = hash + (hash << 10)
   hash = hash Xor (hash >> 6)
  Next
  hash = hash + (hash << 3)   
  hash = hash Xor (hash >> 11)
  hash = hash + (hash << 15)
 
  Return hash
 
End Function

While looking at the original code, I knew that with all the bit-shifting going on, there would definitely be overflows; this algorithm has overflow written all over it. So I anticipated some small bumps, but not real big ones…

OK. I don’t care about bits falling out of the bucket, as long as this doesn’t result in exceptions. How wrong could I be…nearly every line gave me exceptions. But, surprisingly, those exceptions weren’t thrown by the bit-shifting code, but by intermediate results that exceeded the UInt32′s maximum value, for example:

3281982255 + 2085403648 = 5367385903; yep, that’s 33 bits.

OK. Nevermind; I’ll use a UInt64 and only keep the 32 LSBs. That should do the trick. No! The VB.Net OverlowException told me ‘Arithmetic operation resulted in an overflow.’ Yeah I know, with 32 bits it does, but not with 64!
What’s going on here ?. Lets isolate the problem a bit…

Dim a As UInt32 = 3281982255 ' 11000011100111110001001100101111
Dim b As UInt32 = 2085403648 ' 01111100010011001011110000000000
Dim result As UInt64 = a + b

Line 3 also throws the exception I mentioned above. But why?? The result of a+b is 5367385903, which should perfectly fit in 64 bits. This is becoming really weird …
Changing the result type to Decimal, with a largest possible value of +/-79228162514264337593543950335 didn’t help either. Same Exception.

This code works though:

Dim a As UInt64 = 3281982255
Dim b As UInt64 = 2085403648
Dim result As UInt64 = a + b

But why should I have to this; it’s stupid – using 64 bits to store 32 bit values; huh? Since I don’t work with VB.Net that much, I started looking for compiler directives to ignore overflows in this hash routine. The only thing I could find was this:

Yeah, right, remove integer overflow checks… at project level. Who invented that? Seems I can’t use compiler directives in the code at all, and I sure don’t want to remove all the overflow checks from the project of course. I did a little more testing with the Intxx type variables and saw some strange things happening:

Dim a As UInt32 = 1073741824 
Dim b As UInt32 = 1073741824 
Dim result1 As UInt64 = (2 * a) + (2 * b)
Dim result2 As UInt64 = a + a + b + b
Dim result3 As UInt64 = (a + a) + (b + b)
Dim result4 As UInt64 = 5 * a

Line 3 does not result in an Overflow Exception, but line 4 does. Line 5 does too, but line 6 doesn’t. It looks like only additions can trigger those overflows? What is this??

Finally I found out that the UInt32 and UInt64 types are not CLS-compliant and that CLS-compliant alternative for UInt32 is Int64. Again, I don’t want to use 64 bits. I only need 32 of them!

I can’t be the only one having this trouble, so I started searching what this CLS compliance meant. From what I’ve read about this, I know now that .NET doesn’t treat UInt32 values as such internally, which results in the MSB getting back its role as sign bit, which in turn results in a sign bit being shifted out during the arithmetic which eventually results in an Exception…
So .NET doesn’t natively support unsigned data types! OK..

After multiple hours of trying all kinds of stupid things to get the right results without exceptions, I ended up with this code:

Public Function Jenkins_one_at_a_time_hash(ByVal data() As Byte) As UInt32
  Dim hash As UInt32 = 0
 
  For Each b As Byte In data
    hash = ((hash + b) And &HFFFFFFFFL)
    hash = (hash + ((hash << 10) And &HFFFFFFFFL)) And &HFFFFFFFFL
    hash = hash Xor ((hash >> 6) And &HFFFFFFFFL)
 
  Next
 
  hash = (hash + ((hash << 3) And &HFFFFFFFFL)) And &HFFFFFFFFL   
  hash = hash Xor ((hash >> 11) And &HFFFFFFFFL)
  hash = (hash + ((hash << 15) And &HFFFFFFFFL)) And &HFFFFFFFFL
 
  Return hash
 
End Function

So actually, it comes down to ‘manually’ preventing overflows wherever they can occur; even in intermediate results; then why did it take me so long to figure this out???

Hurrying back to Delphi again, yuck!

No more painful toes

This evening I wrote a sketch to protect my toes better. The sketch switches the LED strip on and off based on motion detection by 2 PIRs. It works

The blue LED on the Protoshield indicates when there is motion detected by the PIR. The motion detection turns the LED strip on and it will stay on until a period of 15 seconds with no motion has passed – only then, the LED strip is turned off again.

The sketch takes care of the “soft on/off” feature, by gently raising or lowering the brightness during a configurable time-span.

All that’s left to do is cleaning up the code, solder some wires, wait for the enclosure to arrive and give the LED strip, PIRs and Arduino enclosure a place under the bed.

Simple, yet very convenient automation

Bedroom floor lighting

Last week I had some trouble getting out of bed during the night without hurting myself. So this evening I decided to do something about that; I need something that can light the floor while someone is walking through the bedroom at night. So I made a list of things with which I could make something useful for that:

• An Arduino Duemilanove;
• a Protoshield;
• 2 PIR motion sensors;
• a LDR;
• a  IRLZ34N MOSFET to drive the LED strip;
• a 12V power adapter;
• about 2 m. of white LED strip;
• 2 LEDs (one for power and the other for motion detection);
• an Arduino enclosure.

There it is… our automatic bedroom floor lighting is being tested at this moment. The 2 PIR motion sensors will be mounted under the bed in a way that they will only be able to detect motion caused by moving legs, the LDR will be used to detect whether it’s dark in the bedroom or not and the white LED strips will be glued to the bottom side of the bed and will light the floor when motion is detected.

The fun thing is that this floor lighting is almost completely built from spare parts (except the enclosure). The 2 PIR motion sensors were the first motion sensors I ever bought, but the lenses were too big for my taste to actually use them. Under the bed the size of those lenses doesn’t matter. The Duemilanove is one of the many Duemilanoves I have laying around for testing/experimenting, so I can easily do with one less – I could also have picked an RBBB, Teensy or JeeNode. The number of unused protoshields made me decide for an Arduino. And all the other parts were all purchased in the past with the thought they’d be handy to have around for when you suddenly need them

No RF, Zigbee or Ethernet this time – this will be a solution that doesn’t need any other external input, nor do I think I’ll use the fact that someone’s walking through the bedroom in the rest of my system. Nevertheless, I’ll reserve some space on the Protoshield for a XBee Breakout board cause this would actually be a very good place for a Zigbee router on the 2nd floor!

The sketch will be a collection of code  from other sketches I’m already using, so I hope that at the end of this week I can finish this and never hurt my toes again

Nemef Radaris Evolution and Homeseer

Last summer I became the proud owner of a Nemef Radaris Evolution electronic door lock. It’s in use for 6 months now and has been unlocked almost 1800 times. In that period of 6 months, the Nemef Radaris Evolution has proven itself as a really solid and reliable piece of equipment. I’ve never seen nor heard of a better solution (no, I don’t work there). If I would have written a full scale review about the Radaris Evolution, the conclusion would have been  ”Highly Recommended” – but since I don’t do reviews, you’ll have to do with just the conclusion

Combine the Nemef Radaris Evolution furniture with a Nemef RF Module and you’ve got all the the ingredients to fully integrate the Nemef Radaris Evolution door lock(s) into your Domotica system. However, that’s where things get harder – as far as I know, there’s no (consumer) software available for the connection between the Nemef Radaris Evolution and Home Automation systems. No Plugin, module, app or anything like that to monitor and/or control the Nemef Radaris. Only 3 guys (I know of) have implemented this in their own homebrew system; Pieter Knuvers is one of them.

But this can change very rapidly; I’m working on a Homeseer Plugin for the Nemef Radaris Evolution.

With this Plugin (and a Nemef RF Module (dutch link)) it will be possible to monitor and control up to 4 Nemef Radaris Evolution (dutch link) door furnitures. Opening the door, badge management (yep, you can stow away your programming card), viewing historical data (what badge was used where and when), it’s all in the Plugin. Want to give a badge access only during a certain time period on a certain day? The Nemef Radaris Evolution Plugin and a small script can accomplish just that.

The basis of the Plugin is almost finished now; I already tested the basic functionality by feeding my own historical data to the Plugin and this looks just fine.

Now it’s time to provide the necessary event triggers to Homeseer, so that the Nemef Radaris Evolution door furnitures can really become a part of that bigger picture, called Home Automation.

Back in business

One thing I really hate is when hardware is not working.

Last November and December will be remembered as the “Blue” months, because of the amount of Blue Screens (BSOD) I got. And the problem (with me) is that I just can’t work on anything else, knowing that some piece of hardware can suddenly break down (again). These things consume all my time and energy until it’s fixed.. but I think I’ve got it fixed now!

This time my rather new Hyper-V server was suffering from BSOD’s. It started in November and it got even worse in December; I even had to show the rest of the family what button had to be presssed to get this server on his feet again for when I was not at home.. big fail! Life did still go on of course, but when your complete Domotica system and Internet connection stops working, you realize how much you’re used to  all the “good stuff” it brings and how fast you think you can’t live without it anymore.

This time it was one of the SDRAM modules inside the server that was causing these BSODs. But since I didn’t want to just start replacing hardware components randomly, I had to be sure what exactly was causing these blue screens. So I waited until I had enough BSOD occurrences to justify buying a new set of 4 x 4GB RAM modules. So on December 24th of 2011 I replaced the RAM and the problem disappeared – I’m back in business!

I know I’ll never be able change the fact that my productivity drops below zero when I’m facing these kind of problems. On the other hand, a few years ago I would have thrown this malfunctioning server out the door and start building a new one immediately – but now I take my time, observe where the errors come from and solve the problems in a more relativistic way. OK, I will miss some power consumption “blips”; I have to turn on the lights manually, etcetera – but after all, it’s just a hobby

There is one nice side effect of all this, and that’s that while I was going through all the scenarios to solve the issue, I found a list of components with which I can build a new sub-20W server with even more processing power than I have now. That’s a reduction of 33% in power consumption… sounds tempting!

Energy flow in Watt

Now that I have sensors on all the radiators  I can sit back, relax and watch everything happen. I needed some help of the rest of the family to keep the doors upstairs closed, cause that seems to be very hard for some, but after a few days they all knew that when the doors were closed, daddy was doing one of his experiments again.. yep, this house is one big laboratory

However, doing nothing is not really my style, so I went on reading about radiators, calculations etcetera and found some calculation tools for my RADSON Compact and Jaga Tempo radiators. And although it doesn’t add new data, I thought it would be fun to see if I could use the formulas used in these ‘Installer tools’ (in fact Excel sheets) for myself.

And I was right; based on radiator model, dimensions and specific properties it should be possible to calculate the actual heat output performance of the radiators in other circumstances (temperatures) . If you know the heat output conform the European standard EN442 (dutch) for a specific combination of model & dimensions, the so called n-factor (aka the emission line slope) and the air-temperature, you can calculate the heat output for any combination of flow- and return temperature!

So that’s what I did last evening – preparing my system for these semi-realtime heat output calculations.

First I had to add some more flexibility to the database; creating fields for this purpose only didn’t feel right, so I added a free text field in which I could ‘dump’ anything I want – device specific properties. As an example, here are the properties of one of the radiator devices in my system (it feels like ‘Domotica system’ doesn’t quite cover the capabilities anymore):

TYPE=RC
QN=1407
N=1.3404
TAIR=BATHROOM.TEMP

This means this device:

• is of type RC (as in Radson Compact), 
• has a heat output of 1407 Watt, based on its dimensions and EN442 at 75/65/20 °C (that’s flow, return and air temperature);
• has an n-factor of 1.3403;
• and that the air temperature can be taken from the device value called BATHROOM.TEMP (a foreign key, so to speak)

So every line defines a device specific property, which are dynamically added to the base device class at run-time.

And I had to write 2 Delphi functions to do the math, primarily based on these formulas:

Q is the variable that needs to be calculated, the rest is all known – piece of cake!

Just make sure you take care of unexpected results, like divisions by zero or raising negative values to a power- working with live data can produce strange results, where a radiator can suddenly start cooling instead of heating

I added the calculated heat output (the “W” column) in the table with all the other sensor data as can be seen below:

Cool! Although these heat output numbers are based on pretty accurate formulas and properties provided by the manufacturer, the real-life numbers will probably be lower. Dust, bad airflow caused by windowsills and such will always result in deviations. How much? Dunno…

RF to Zigbee gateway

The last piece of missing hardware is finished. The picture below shows the 2nd RF to Zigbee gateway I had to make to be able to receive all the Hydronic balancing sensors I made earlier this week. One of those sensors just couldn’t make it through 3 walls all day long, so I created  a temporary solution on a breadboard to solve this.

A very simple yet effective way (for me) to get the sensor data where I want it (in my Domotica system) with minimal effort.

The JeeNode acts as a RF receiver and just echoes everything with a valid CRC to the Digi XBee; from there it eventually arrives at my Zigbee Coordinator with which I can communicate over TCP/IP.

The JeeNode runs a slightly modified version of the RF12Demo sketch made by Jean-Claude Wippler. I used the NewSoftSerial library to create an additional Serial port, and added a few print statements for the XBee port, right there where the RF12Demo Serial.println()’s the received RF data to the Serial port. Compile, Upload, setting the RF band,  group- and node ID and I’m done!

This JeeNode is powered by a 5V USB adapter and the XBee gets its power from the 3.3V JeeNode ports. The XBee uses a Zigbee End Device AT firmware (2864) with the Sleep Mode set to Pin Hibernate. But because pin 9 is wired to GND, this means that the XBee is permanently on.  Only 3 wires are needed to connect the XBee to the JeeNode: 3.3V, GND and a JeeNode digital pin to the XBee DOUT.

That’s it – moving on with where this all started with: understanding the flow of  heating energy in our house!

HB sensors finished and installed

Almost ready to start investigating hydronic balancing (HB) …

Everything is in place and working except for 1 sensor, which is too far away from the receiving JeeNode (which forwards all the data to my Domotica system). So I’ll have to build another RF receiver for that; the missing parts will arrive tomorrow I hope. In the meantime I started creating a webpage to display all the information I want to observe, so I don’t have to switch between browser tabs and go from one page to the other. Another thing that has to be done is program the thermostat, so that the boiler will have to burn at full power for a few hours every day, so that I can see what happens.  I’ve already seen some things that need to be adjusted!

It’s them floats again..

I keep a close eye on a lot of things, including my own Domotica system. Not just things like empty batteries in the sensors, but also the software behind it all. Things like exceptions that occur, memory usage, cpu usage and stuff like that. It keeps me informed about the ‘health’ of it all, so I won’t end up with a system with lots of memory leaks or other nasty bugs scattered throughout the system; and which will be much harder to find once it crashes due to not enough memory…

Yesterday I saw something strange happening to the memory usage:

Immediately after I upgraded my system, memory usage started to increase to values that I had never seen before. It’s always a straight line with a variance of +/- 2 KB after 24 hours of up-time.

Strange, cause all I had done was adding a few lines of code to support the hydronic balancing sensors; nothing exciting actually. I checked if all objects were destroyed after use; nothing wrong. I did a restart; same thing happened again. The only thing the code had to do was calculate 2 temperatures out of 4 integer values; what could be wrong here?

Digging a bit deeper I saw these messages in the logs, a few per hour:

Exception:Database Server Error: SQL State: 01000, SQL Error Code: 3621

Not very self-explanatory either, but it was evident that something was wrong here. Again strange, since SQL Server has been running without any problem since I use it; that’s 5 years! I looked up the SQL command that triggered this error response and there it was… a string value of ’0.689999999999998′ was written to a fixed-length field and it didn’t fit!

But where does this ’0.689999999999998′ come from? Working with floats of course!

The 2 temperature values are each calculated by dividing a word (2 bytes) by 100; this results in a float with a maximum of 2 decimals (duh). But by storing these 2 temperatures in float variables and subtracting those 2 (to calculate the temperature difference), the result of 28.06 – 27.37 wasn’t 0.69 but the error triggering 0.689999999999998…

Of course, I know this and I should have anticipated on that – I forgot to round the result of the subtraction to 2 decimals!