Sunday, August 25, 2013

How to pimp a greenhouse

My wife got a luxury greenhouse this spring and while I certainly enjoy the crops coming out of it, it's not exactly aligned with my own geeky hobbies. While unfortunately it's not in the cards for me to invest in a solar system generating green electricity for my entire household, I *can* at least construct a system for a 14m2 greenhouse. The goal is to build an off-grid system for lighting the greenhouse at night using energy provided by the sun in the day, as cheaply as possible.

Sourcing components

There are many small solar system packages complete with panel, charger and battery; but the price typically starts up around US$200 and that's without the light. For me, part of the fun in something like this, is doing it yourself with cheap COTS components. After some browsing around, ordering and waiting for things to arrive, I ended up with the following:

2x 5W amorph solar panels from local hardware store (US$45).



No-name Chinese 10A MPPT charge controller from ebay (US$10).



12V, 4Ah AGM scooter battery from local hardware store (US$25).



2m RGB LED tape, 30 LED per meter, IP67 from ebay (US$25).



RGB LED driver with wireless controller from ebay ($15).


Lots of things were in the open here. Perhaps the panels would not be good enough to charge the battery, perhaps the charge controller did not really use MPPT, perhaps the battery would not be large enough to hold the power needed by the LED's etc. While I did some preliminary calculations, in practice things might not always pan out the way you think, so I bought the parts primarily because they were cheap and would appear to do the trick - I figure I could always improve insufficient aspects later on. Total cost landed around US$120 - not too bad at all.

Preliminary checks

I received the charge controller about 10 days after ordering, which is pretty typical from China to Denmark. It felt really light and cheap, so I popped the hood to take a look at it inside.



Ok well, we're not seeing any sign of inductors for the DC-DC step. I think I see a quad op-amp (battery level monitoring?), some Schmitt triggers (hysteresis?), 3 noname power FET's (chargers), a few transistors and a bunch of discrete components - but definitely nothing to convince me this is an MPPT charger. This was confirmed by measuring the voltage over a connected panel, noticing that the voltage got pulled down to that of the battery, *not* kept at the maximum power-point which I determined manually to be around 18V.


Had I payed more for the device, I might have been disappointed, but at US$10, even a naive PWM charger is a good deal. The charger is rated for 10A, but I will barely be pushing 1A, so the circuit won't be particular stressed at all.

At maximum current draw (white color, all LED's turned on), the 2m LED strip consumes 0.7A/8.4W, and with a single color lit, it consumes 0.4A/4.8W (on average since blue uses more than green which uses more than red) so I chose to test the battery by putting a dummy load on using a computerized RC charge/discharge controller I had laying around.



It's worth nothing, that the battery was not actually full when I started this test, I had probably discharged it some 10%. With that in mind, the relatively small 4Ah battery still manages to provide 4½h of 500mA before it drops below 11.5V - which represents a relatively deep discharge cycle down to around 40% with 1.7Ah left. That should be good enough for the LED lighting to last all evening. The charge controller draws just 15mA when idling (not charging nor discharging the battery) so that shouldn't be a problem either.

Installation

Using some spare aluminum bands, I constructed an element that fits perfectly between two of the support rods of the greenhouse.



During the construction, I tried using various sized wires and actually came short of some decent 4-pole wire with enough copper to conduct the current without too great a loss in the wire. My original idea was to have controllers and battery some 2-3 meters away from the solar panels and LED light, but that would've resulted in about 1W being dissipated in the wires. So instead, I attached the LED driver and the charge controller directly to the back of the constructed element, losing an order of magnitude less power. The battery I hung in an aluminum band, which got bend to clamp perfectly around the battery and hold it firmly in place. I also left space for a second battery which I might add if capacity becomes an issue.



Testing

After placing the 2m LED strip to the rim of the ceiling using the adhesive tape on the back, it was time to light it up. Apart from a small mix-up between two of the colors, everything worked perfectly. However, it was not until nighttime it truly came to its right.







Admittedly, some of the colors can look a bit corny and signal "red light district", but toned down a bit it adds to a very cozy ambiance in the yard at night - I'm especially fond of the warm-yellow/orange and green/teal glow. Of course, it can also do plain white. With the brightness seen in the above pictures, the LED tape uses about 0.5A/6W. It's a testament to the efficiency of LED lighting, that I worry if it's too bright late at night, but I will have to wait and see whether the neighbors complain or not. ;)

Conclusion

I've learned, that when playing around 12V systems with currents approaching 1A, keeping wires short (or using excessively thick ones) seems paramount for efficiency. It would've been nice if I could have afforded the Philips Hue system, which can be controlled from any smartphone and is hackable to do cool things like adapting the brightness to the time of the day, flash blue when there's a Facebook message etc., but I really find this Philips product too expensive for what it is, so chose the cheap approach instead. At a mere US$120 I am more than happy with the solution and so is the wife btw. Time will tell if I need a better panel (30W mono panel can be had cheap on ebay) or an extra battery. One obvious improvement I will probably look into, is adding a dusk-relay, to ensure the LED strip can only be lit when it's dark enough, to prevent the LED's from stealing juice from the solar panels during the daily charge cycle.

Update

After a few days of testing, everything continues to work as intended. The greenhouse is typically lit from around 21:00 when it starts to get dark, to midnight. If I were to guess, I'd say the LED could stay on most of the night, but have yet to actually try this (and thus, test the load cut-off feature). Interestingly, on a late-summer day at the end of august, the system seems to stop charging in the early afternoon where it reaches 13.25V. This is promising, as it suggests being able to catch up on more gloomier days, but over a longer period.



Tuesday, August 6, 2013

Talking to a Kamstrup 685-382 electricity meter

My utility company provides me with ways to see historic consumption of electricity up to a few days ago. However, by that time I have long forgotten when, what and why I did to consume as I did. In order to save power and money on the utility bill, one needs to have some way of monitoring and discovering usage patterns *immediately* as they take place! When I saw you could buy cheap US$50 used industrial strength electricity meters in the form of the Kamstrup 685-382, I decided to buy a few of those. The idea was to have it installed as a secondary meter in my house and try to hook up some sort of communication interface, connected to a low power computer responsible for data acquisition, analysis and presentation.

It has to be said up front, that I am far from the only one looking into this. On a Danish engineering discussion forum, I came by other people experimenting with the Kamstrup 382, except that none of the info I came by there seemed to apply to my version of the meter. So be advised, there must be several (incompatible) versions of the 382 in production and what I write below, may or may not apply to your version of the meter!

The Kamstrup 685-382 meter

Kamstrup is a Danish company with a primarily focus on designing, manufacturing and sale of metering solutions to utility companies around Europe. One of the more dominant meters seems to be the 382, which appears to come in countless versions and revisions, and is still being sold. The picture below shows the two older ones I got a hold of.


Physical characteristics of the 382

The meter is a 3-phased electronic meter, capable of measuring voltage, power, consumption etc. for your entire household. It can be augmented using expansion cards like wireless M-bus etc., but in a vanilla version, it only has an optical input/output port and this is the one I am going to focus on in the following.


The optical interface

Somewhere in the 80's, these kinds of industrial meters started getting support for an IR optical interface, then known by IEC61107 classification and later by IEC62056-21. This optical interface would allow direct RS232 (legacy low-bandwidth serial port) communication with the outside world, primarily for maintenance and service purposes. There are many ways to get a hold of such a physical connector, they can be purchased online (be hold, Kamstrup wants US$230 for theirs) or made from scratch if you can handle a soldering iron.



After shopping around for a while, I decided I was not willing to shell out that much money on so few components, so I started playing with components on a breadboard, hooked up to an USB-to-Serial interface. However, then I found this little gem from an old Elector magazine, and hooked it up, it worked right off the bat. I would play with simpler versions (electrically the rs232 physical requires negative voltages and is not directly compatible with classic 3.3V/5V TTL levels) but this design was the first that I build up (who doesn't have an 741 op-amp?!) into an actual physical connector, so it will remain what I shall use throughout this blog entry.


In order to test the hardware, I tested using the simple IEC61107 standard, which the Kamstrup 382 were supposed to support. Now, one thing to note when you do your own physical optical reader is that you might very well experience cross-talk, that is, receive the very same data bits you are sending. This happened in my case anyway, and the solution was to lower the sensitivity of the receiving diode and/or the transmission power of the emitting diode (thanks for the tip PHK). Since we are dealing with half-duplex synchronous communication, I added cross-talk compensation to the software, so it can work between with flaky DIY optical interfaces.


Yay, so after confirming the hardware works, we're ready to play! Next up, KMP connection and protocol stuff.

Kamstrup's definition of "openness"

Several places in Kamstrup's material (PDF example), they refer to their KMP protocol as being open:
"12.1.2 Open data protocol
Companies who want to develop their own communication driver for the KMP protocol can order a demonstration program with "open source code" in C# (.net based) as well as a detailed protocol description (in English language)."

Notice that this small section, in no uncertain terms, mentions "open data protocol" as well as "open source code". However, after bouncing a few emails back and forth with Kamstrup, it turns out that their understanding of "open" differs somewhat from the established official meaning of the concept:.
Open-source software is software whose source code is published and made available to the public, enabling anyone to copy, modify and redistribute the source code without paying royalties or fees.

Kamstrup told me that I would have to ask permission from my utility company and sign an NDA, before they could hand any software or protocol documentation over to me. Now, a protocol or API can not be copyrighted or patented, so my bullshit alarm got triggered with a strong suspicion that Kamstrup is a practitioner of security through obscurity... it would later become apparent why Kamstrup might have gotten tempted by this erroneous strategy (hint: bad bad security).

Protocol sniffing

So, since we get no help from Kamstrup, we're just going to have to sniff out the protocol ourselves. This is not trivial and requires 3 things; something to sniff, a sniffer tool and a whole lot of patience and thinking.

What to sniff

Even if I now own several of Kamstrups meters, purchased used, I was pretty sure they were not going to provide me with the software that goes with the meter and allows for reprogramming and data acquisition. Nowhere on Kamstrup's official website is there a download link for software or the like, so once again we appear to be dealing with an obscurity thing. However, in one of their published documents I was able to find information about a software utility referred to as MeterTool. On page 6 in this document, there's even an FTP address with credentials information leading into Kamstrup's software library. I downloaded and installed the application on an old Windows laptop and tried connecting to my meter via the optical cable, and bingo, it worked!


Sniffer tool

There are countless ways to sniff the traffic on an RS232 port. You can just guess your way around when it comes to connection settings, but I took a look at the waveform with a cheap oscilloscope (find the duration of a bit and take the inverse value to get a frequency which again is easily converted to baud/bps). The connection settings used were 1200 baud, using 8 data bits, even parity and two stop bits.




Very few people (incl. me) have a digital storage oscilloscope with enough memory to capture and interpret live RS232 data capture, so I ended up using a freeware/shareware serial port monitor to capture entire communication sessions while performing some defined action (shaking hands, getting data, logging in etc.) in the MeterTool utility.

Patience and thinking

Ok this was the hard part, and the part I never really finished and probably never will. After recording various traces with the sniffer utility and noticing the values from the MeterTool, the detective work could commence. At first, it didn't make much sense to me at all. The protocol was not ASCII based (like IEC61107) and it quickly became obvious that it was not simply enough to look for various word sizes and interpret these as binary blocks either.



Entropy and mutation analysis

Eventually I realized that the password when logging in (to reprogram stuff) had to be between 0 and 65535 (thus, 2 bytes in size) and when I did successive traces trying "1", "2", "3"... and so forth, a pattern started emerging. I noticed that a certain byte was decreasing, while another one (at the end) increased. Exploring the boundaries of this 2 byte range, told me that the last two bytes were probably some kind of check-sum and the stuff I saw mutating in the middle of the byte stream, were probably the raw data itself. The funny thing was, I did not see just 2 bytes mutate as one would expect with a value range between 0-65535, instead I saw 4 bytes mutate. From then on, I had a strong suspicion that I had to interpret everything by double-byte.

Another odd thing I noticed about the data, was the limited entropy associated with most of the bytes. Most bytes would hold a value between 0x30 to 0x46 (16 different mutations), indicating that a logical byte was split between two physical bytes on the wire (16 * 16 = 256). This corresponded nicely with the former findings; it explained why I saw 4 bytes mutating when poking inside a 2-byte range and it also meant that the two byte check-sum was really only one logical byte.

Knowing a little about the difference between two physical and one logical byte, meant that now it was somewhat easier to try and interpret, coming up with a mapping mechanism between the MeterTool and the meter. Indeed, this was trivial when looking at some of the trace data from the "authorization" attempts from before. Turns out that that if one interprets the bytes in pairs as ASCII characters, it corresponds to the 4 MSB's and the 4 LSB's of a logical byte:

Physical byte Logical 4-bit nibble ASCII
0x0 0x30 '0'
0x1 0x31 '1'
0x2 0x32 '2'
0x3 0x33 '3'
0x4 0x34 '4'
0x5 0x35 '5'
0x6 0x36 '6'
0x7 0x37 '7'
0x8 0x38 '8'
0x9 0x39 '9'
0x1a 0x41 'A'
0x1b 0x42 'B'
0x1c 0x43 'C'
0x1d 0x44 'D'
0x1e 0x45 'D'
0x1f 0x46 'E'


As you can see, in spite of having a full 8-bit binary data-link layer available, Kamstrup divided a byte into two logical 4-bit (0 - 15) nipples, and lets each nibble be represented by using a full physical byte from the ASCII alphabet 0123456789ABCDEF.

Example: Two physical bytes of 0x31 and 0x32 would map to the ASCII characters '1' and '2'. These are to be interpreted as the upper halv of a hex byte and the lower halv. So '1' becomes 1, l-shifted 4 times (or multiplied by 16) and then '2' becomes 2 which is added to 16, making 18.

This is a peculiar way of representing data, and I have no idea why it's done like this... if you recognize this encoding, please let me know. :)

Time to write some software


My end goal is to have a small and cheap logging solution, analyzing various consumption aspects of my household, so a cheap Android stick which I've written about before, seems like an obvious target. This suggested that I write some software using Java. Though professionally I write Java almost every day, that language is a particular poor choice when it comes to near system-level programming dealing with raw bytes (no support for unsigned bytes!). Furthermore, for unfathomable reasons, SUN Microsystems never bothered to include serial support into the JRE or JDK. Since raw C seems a unnecessarily low level in comparison, and how C# for all practical purposes is a nice middle-ground, I chose that as an implementation language. Yeah I know, purists will yell Microsoft fan-boy at me, but in my opinion C# is a nice iterative non-ivery-towerish improvement over Java. On top of that, it's an open standard, it has great Android support, provides support for unsigned bytes as well as an encapsulated rs232 API.

So in order to work with the odd byte mapping we discovered before, we can write some mapping methods. This is what I came up with:

 protected byte ToKamstrupValue(byte value)
 {
  Debug.Assert(value >= 0x0 && value <= 0xA);
   
  return (byte)(value + (value < 10 ? 0x30 : 0x37));
 }
  
 protected byte FromKamstrupValue(byte value)
 {
  Debug.Assert(value >= 0x30 && value <= 0x46 && value != 0x40);
   
  return (byte)(value - (value < 0x3a ? 0x30: 0x37));
 }

 protected byte[] ToKamstrupPair(byte value)
 {
  byte[] quad = new byte[2];
   
  quad[0] = ToKamstrupValue((byte)((value >> 4) & 0xf));
  quad[1] = ToKamstrupValue((byte)(value & 0xf));
  
  return quad;
 }
  
 protected byte FromKamstrupPair(byte[] pair)
 {
  Debug.Assert(pair.Length == 2);
   
  return FromKamstrupPair(pair, 0);
 }

 protected byte FromKamstrupPair(byte[] pair, int startOffset)
 {
  Debug.Assert(pair.Length > 1);
   
  return (byte)((FromKamstrupValue(pair[startOffset]) << 4) | (FromKamstrupValue(pair[startOffset+1])));
 }



Comparing payload content vs. the two checksum bytes, hinted at a relationship; whenever a byte value in the payload increased, one of the two checksum bytes would decrease. Furthermore, the entropy (or lack thereof) suggested use of the same nipple mapping as described earlier. The problem just got reduced to finding a checksum algorithm resulting in a byte and it wasn't long before the right candidate was found to be 8-bit LRC (Longitudinal Redundancy Check):

    protected virtual byte Checksum(byte[] data)
    { 
        byte checksum = 0;

        foreach(byte value in data)
        {
            checksum += value;
        }

        return (byte)((checksum ^ 0xFF) + 1);
    }


For ease of use, I have encapsulated data-link aspects behind an IMeterConnection interface, and interpretation aspects behind an IMeterProtocol interface. It then becomes very easy to use the library to talk to the meter:


    using(var connection = new SerialMeterConnection{
            PortName = "/dev/ttyUSB0",
            BaudRate = 1200,
            Parity = Parity.Even,
            DataBits = 8,
            StopBits = StopBits.Two,
            Handshake = Handshake.None,
            Encoding = new ASCIIEncoding(),
            NewLine = Encoding.ASCII.GetString(new byte[]{SerialMeterConnection.LF})
    })
    {
        using(var protocol = new MeterProtocolKMP382(connection, logger))
        {
            foreach(var entry in protocol.Registrations)
            {
                Console.WriteLine (entry.Key + ": " + entry.Value);
            }
        }    
    }



A small video demonstrating the conversation between the software and the meter:



Conclusion

It was a fun exercise, but it took a long time to figure out how to interpret the data. Meanwhile, I have had ½ a blog entry sitting idle for at least a year - the eternal problem of a tinkerer with a day job. The relevant driver source code has been released under a BSD license available through GitHub, for others to play with. The most interesting part probably being MeterProtocolKMP382.cs. This is just the underlying driver with runnable example code, the source for my complete Android meter acquisition app is not anywhere close to being done and next step is to attack my two other Kamstrup meters (water and heating).

Now back to the rather abysmal security aspect. A two byte security code is obviously not enough in this day and age, indeed it takes just about 24h to brute-force your way to the security code of this meter. This code, as far as I know, gives you the option to reprogram the meter in a variety of ways - meter no., customer no. etc. I have chosen not to include this brute-force code even if it is just a simple loop waiting for a timeout to continue trying next password. I used this approach to guess the code of my meters (12345, which appears to be the default), but I'll leave that as an exercise to any reader who would be interested (you can find a LOGIN command in source code).

As always, feel free to comment and/or fork the project, adding more features. There *are* fields which I have not been able to interpret and I have not sought to investigate programming commands, although the process is the same as described in the above.