DEMUX Chaining

Add Comment | Nov 14, 2010

There are several ways to chain demux IC’s to get more outputs than the ones that a single IC can get you. In these posts I’ll explore the ones I could find and use successfully. The first one has to do with the differences of inverting and non-inverting IC’s. In this video, I have 2 iC’s driving 16 LEDs. With this configuration I could drive as much as 64 LEDs with these 2 IC’s. I simply didn’t have the patience to connect them all:

16 LEDs with 2 IC’s

In my previous post I mentioned that the 74HC138 should be connected to the LED via their cathode, meaning that the IC is effectively working as ground for the LED. That’s because the 138 is an inverting 3-to-8 demux. The 74HC238, on the other hand, is not inverting so when you connect it to the LED, the LED is connected to the IC via its anode. The IC is providing power to the LED, which is connected to ground normally.

We can use this characteristic of the ICs to connect 64 LEDs using one of each. Think of it as a table, where you have 8 by 8 LEDs. The “rows” are connected to the 138 and the “columns” are connected to the 238. Only when the 138 is coding a LOW and the 238 is coding a HIGH, the LED will light. Here’s a schematic for the 16 LEDs I connected:

16LEDSPOT

The 138 has its first two output pins connected to 8 LEDs each, and the 238 has its output pins connected to 2 LEDs each. For 64 LEDs, you should connect the 8 output pins in each IC to 8 LEDs. Notice that since I’m only using one input in the 138, I have connected the other inputs to ground. This effectively codes a LOW for all those inputs. The enablers are hard wired to activate the IC’s.

In this schema I also have a potentiometer. It looks like this:

16 LEDs, using a potentiometer

The potentiometer is connected to A0. I also connected AREF to the power line in the breadboard, so it will have the same amount of current as the potentiometer (without the resistance, that is). You must connect the AREF to your maximum when using an analog pin. The rest of the schema, I think, speaks for itself.

So what does the code look like? The encoder is exactly the same as the one I’ve been using for the last couple of posts, but this time initialized with 4 digital pins. Again, if you want 64 LEDs you’ll have to initialize it with 6 digital pins:

Encoder =

new Demuxer(Pins.GPIO_PIN_D0, Pins.GPIO_PIN_D1, Pins.GPIO_PIN_D2, Pins.GPIO_PIN_D3);

I also have the potentiometer coded as an AnalogInput:

Potentiometer =

new AnalogInput(Pins.GPIO_PIN_A0);

The AnalogInput must be initialized with the range to work correctly. I have it initialized like this:

Potentiometer.SetRange(0, 15);

So my breadboard class now looks like this:

1 public class BaseBoard 2 { 3 public Demuxer Encoder { get; private set; } 4 public AnalogInput Potentiometer { get; private set; } 5 6 public BaseBoard() 7 { 8 Encoder = new Demuxer(Pins.GPIO_PIN_D0, Pins.GPIO_PIN_D1, Pins.GPIO_PIN_D2, Pins.GPIO_PIN_D3); 9 Potentiometer = new AnalogInput(Pins.GPIO_PIN_A0); 10 11 Potentiometer.SetRange(0, 15); 12 } 13 }

And the main program looks like this:

1 public static void Main() 2 { 3 board = new BaseBoard(); 4 5 while(true) 6 { 7 board.Encoder.Encode(board.Potentiometer.Read()); 8 Thread.Sleep(100); 9 } 10 }

Simple, isn’t it?

There’s a huge disadvantage in chaining the IC’s this way: you must provide power and ground. That is, if you wanted to create a shield that provides 64 outputs (or 1024, using all the digital pins in the Netduino) you simply can’t use this technique. There are other ways to do it that I’ll explore in the next few posts. On the other hand, this technique is really simple to wire, uses only 2 ICs (so it’s cheap) and works great.

DEMUX with Netduino

Add Comment | Nov 06, 2010

In my previous post I used a lot of transistors (not very successfully, I might add) to drive 4 LEDs with 2 digital GPIO pins from a Netduino. I mentioned that there are Integrated Circuits (ICs) that may be used for that. In this post I’m going to use one of those ICs (specifically, a 74HCT138) to do the exact same thing. First, here is how it looks:

Very simple prototype of the use of a 74HCT138

 

The code I’m using is exactly the same as before. I have the Encoder class, which will receive a single int value an encode it in binary with as many pins as I provide:

1 public class Encoder 2 { 3 private OutputPort[] outputs; 4 5 public Encoder(params Cpu.Pin[] pins) 6 { 7 outputs = new OutputPort[pins.Length]; 8 for (int i = 0; i < pins.Length; i++) 9 { 10 outputs[i] = new OutputPort(pins[i], false); 11 } 12 } 13 14 public void Reset() 15 { 16 foreach (var port in outputs) 17 { 18 port.Write(false); 19 } 20 } 21 22 public void Encode(int code) 23 { 24 for (int i = 0; i < outputs.Length; i++) 25 { 26 outputs[i].Write((code & ((int)System.Math.Pow(2, i))) > 0); 27 } 28 } 29 }

This Encoder will be useful for the next few posts too. I declare the Encoder in my Baseboard using digital outputs 0 and 1 (line 9 in the code):

1 public class BaseBoard 2 { 3 public Button Button { get; private set; } 4 public Encoder Encoder { get; private set; } 5 6 public BaseBoard() 7 { 8 Button = new Button(Pins.ONBOARD_SW1); 9 Encoder = new Encoder(Pins.GPIO_PIN_D0, Pins.GPIO_PIN_D1); 10 } 11 }

And in my program, I simply encode the next number when the switch is released:

1 public class Program 2 { 3 private static BaseBoard board; 4 private static int code; 5 6 public static void Main() 7 { 8 code = 0; 9 board = new BaseBoard(); 10 board.Button.Released += new NoParamEventHandler(Button_Released); 11 12 board.Encoder.Reset(); 13 14 Thread.Sleep(Timeout.Infinite); 15 } 16 17 static void Button_Released() 18 { 19 code = (code + 1) & 3; 20 board.Encoder.Encode(code); 21 } 22 }

And that’s the code. I have a longer explanation of how the code works in my previous post if you’re interested. What I’m interested in this post is the hardware side of things.

The 74HCT138 is a 3-to-8 DEMUX, Inverting. That means that it can turn 3 digital outputs into 8 digital outputs. Inverting means that the encoded output will actually be LOW, not HIGH as would be expected. That means that if you encode, for instance, a 4, the 4rd pin (Y3, as this is zero based) will be LOW while pins 0, 1, 2, 4, 5, 6 and 7 will be HIGH. For the purposes of this demo, Inverting and not Inverting are exactly the same. I’ll show you how to wire the non-inverting version at the end of the post. This circuit has also a fantastic characteristic: it has 3 enable inputs. The circuit will only activate if E1 and E2 are LOW and E3 is high. This is used for chaining, which is something I’ll explore in a later post.

So let’s go to the wiring. The project in the video looks something like this when wired:

4LEDS-138

It doesn’t show clearly in the picture, but the marking of the chip is facing left. The chip has 16 pins, 8 per side. The right side of the chip holds Vcc and pins Y0 to Y6. The left side holds A0, A1, A2, E1, E2, E3, Y7 and GNC. The pins marked as Yx are the encoded outputs. The ones marked with Ax are the selectors (in this case, the value to encode) and the ones marked with Ex are the enablers. If you check the datasheet you’ll notice that E1 and E2 have a little bar over them. That means that the input needs to be LOW for them to work.

The wiring is pretty straightforward. Vcc is connected to Netduino’s 3.3V and GND to Netduino’s GND. A0 is connected to D0 and A1 is connected to D1. A3 I’m not using (I only have 4 LEDs). I could use it if I where connecting up to 8 LEDs. Since I’m not using it, I have to connect it to GND so it’ll always be LOW. E1 and E2 are also hardwired to LOW and E3 is hardwired to HIGH. Simple, isn’t it?

As per de LEDs, since I’m using an inverting IC I have to do something that’s counterintuitive: the LED’s are connected to the power line in the breadboard via their anode (the long leg) and to the output pins in the IC via their cathode (the short leg). That is, the IC is working as ground for the LEDs. When the output is HIGH, the LED will not turn on, when it’s LOW it will.

Let me show you how this works in a 74HCT238:

4LEDS-238

If you look closely, you’ll notice that the only difference is that the leds are connected to ground via their cathode and to the IC via their anode. That is because the 74HCT238 is not inverting. When you encode a 4, the 4th pin will be HIGH and all the other pins will be LOW. Even the enablers work exactly the same in both ICs. This difference will let us connect 64 LEDs with only 2 chips. I’ll do that in a later post.

A single IC can turn 3 digital outputs into 8 exclusive digital outputs. But these ICs can be chained in many many ways, and I’ll explore the ones I thought of in the next few posts.

Off topic: The soft I’m using for the images is open source and extremely cool. It’s called Fritzing, and it lives on donations and on selling an Arduino UNO starter kit I’m going to buy before year’s end. If you are planning on buying one, support them by buying from them! I’m not related to that project at all but I’m using it so much to write these posts that I feel that I owe them.

UPDATE: Reading the datasheet closely, I found out this ICs work only as DEMUXs. I updated the post to remove the references to MUXing.

Magic Transistor Boolean Logic

Add Comment | Nov 03, 2010

WARNING: I’m probably doing something terribly wrong here.

In my last post, I mentioned that using transistors you can do boolean logic. This is the fundamental basis for computers to work. I wanted to test the concept. In that article, I was using a single Netduino digital output to drive two LEDs. In this experiment, I’m using two digital outputs to drive four LEDs, using some transistors. Here’s the video:

Driving 4 LEDs from 2 pins using transistors

 

You can see the LEDs turning on in order when I press the switch. Following my first post, I coded an Encoder. The Encoder receives a single int does some boolean operations and turns it into outputs in the pins. The code is as follows:

1 public class Encoder 2 { 3 private OutputPort[] outputs; 4 5 public Encoder(params Cpu.Pin[] pins) 6 { 7 outputs = new OutputPort[pins.Length]; 8 for (int i = 0; i < pins.Length; i++) 9 { 10 outputs[i] = new OutputPort(pins[i], false); 11 } 12 } 13 14 public void Reset() 15 { 16 foreach (var port in outputs) 17 { 18 port.Write(false); 19 } 20 } 21 22 public void Encode(int code) 23 { 24 for (int i = 0; i < outputs.Length; i++) 25 { 26 outputs[i].Write((code & ((int)System.Math.Pow(2, i))) > 0); 27 } 28 } 29 } 30

The boolean logic at work is fairly simple, but just in case, this is what I’m doing:

For each pin, I’m computing 2^pinNumber, and then ANDing that to the value to encode. If I have for instance the value 5, that is represented by 101 in binary.

2^0 = 1 = 001 in binary

2^1 = 2 = 010 in binary

2^2 = 4 = 100 in binary

Doing the AND operation with this values, I can see that 5 will be encoded by turning on the first and last pin, and turning on the middle pin (which will represent 101, and is what I wanted to begin with). The maximum number I can encode is 2^numberOfPins – 1. With 2 pins, as I’m using in the sample, I can encode ( 2^2) – 1 = 3, so I can encode the numbers 0, 1, 2 and 3.

The Program then is very simple. I have a static int holding the current code and increment that when the button is released. The increment is constrained by 3 by using a boolean trick that only works for powers of 2, which is ANDing with the maximum possible number. Here’s the code:

1 public class Program 2 { 3 private static BaseBoard board; 4 private static int code; 5 6 public static void Main() 7 { 8 code = 0; 9 board = new BaseBoard(); 10 board.Button.Released += new NoParamEventHandler(Button_Released); 11 12 board.Encoder.Reset(); 13 14 Thread.Sleep(Timeout.Infinite); 15 } 16 17 static void Button_Released() 18 { 19 code = (code + 1) & 3; 20 board.Encoder.Encode(code); 21 } 22 23 } 24

The board, as you’ll expect, simply declares the Button and the Encoder:

1 public class BaseBoard 2 { 3 public Button Button { get; private set; } 4 public Encoder Encoder { get; private set; } 5 6 7 public BaseBoard() 8 { 9 Button = new Button(Pins.ONBOARD_SW1); 10 Encoder = new Encoder(Pins.GPIO_PIN_D0, Pins.GPIO_PIN_D1); 11 } 12 }

And that’s about all I need to tell you about the code.

On the hardware side I’m doing something (that is probably wrong) that involves using NPN transistors as an identity function of sorts and PNP as a negate instruction. That is, if an NPN receives a HIGH from the output it activates, and when a PNP transistor receives a LOW from the output it activates. So 0 (which is 00 in binary) will use two PNP transistors, the 1 (01) will use a NPN and a PNP transistors, the 2 (10) will use a PNP and a NPN transistor and the 3 (11) will use 2 NPN transistors. The problem with my design (I think) is the position of the LED and the current relating to the transistors. You will notice that when the rightmost LED is on, the next LED to the left is partly on, which is wrong. Here’s a nice schematic for the design (using an Arduino, but the Netduino is pin-by-pin identical in this case). I’m only using 2 2k resistors for the transistors (I’m using a lot more in the video) but it works exactly the same:

ICTransistor

Of course the correct way of coding this is using a MUX, and I’ll explore that in my next post.

Netduino and Transistors

One Comment | Oct 23, 2010

Be warned: this will be very basic. I find it amazing nonetheless.

As a newbie, I went out and bought a 5V led to be used in an output port in the Netduino. This will of course not work, as the output ports of the Netduino are 3.3V. “No problem”, I thought, “I’ll just use some resistors”. The only problem is that resistors don’t work that way. You can take 5V to 3.3V by using resistors (although you could take 5V to a 3.3V led). I ended up giving the led to a friend who does very cool led projects, but I couldn’t let it rest. The point is that the Netduino does have a 5V source. So I did a little research on how to use transistors for switching, and this is the result:

Two blinking leds, using transistors as switches

Most of the data I got from this excellent pdf. For this experiment I used 2 BC547B transistors, and 2 2K resistors. The resistors are very easy to calculate in this case. I think anything from 1K to 5K will work. The only point (as is usual with resistors) is not to burn another component, in this case the transistors. The code for the project (based on my last post), is simply this:

1 board.GreenLed.TurnOff(); 2 board.RedLed.TurnOn(); 3 4 while (true) 5 { 6 board.GreenLed.Invert(); 7 board.RedLed.Invert(); 8 Thread.Sleep(1000); 9 }

The code for the board is this:

1 public BaseBoard() 2 { 3 GreenLed = new Led(Pins.GPIO_PIN_D6); 4 RedLed = new Led(Pins.GPIO_PIN_D7); 5 } 6

The transistors have 3 pins: Collector, Base and Emitter. In the transistors I got, they are in that order, from left to right (when the transistor is facing you with the written flat surface). These are NPN transistors. That means that when the base has current (as little as 1mA), the collector and emitter are connected. In PNP transistors, it works the other way around (when the base doesn’t have current, the base and collector are connected).

The connection is the same for both leds: connect the emitter to ground, the collector to the cathode (the short pin) of the led and the base to the resistor. The other leg of the resistor should be connected to the output pin in the Netduino, and the other pin in the led (the anode) to the source (5V or 3.3V, as needed). And that’s it! With that, you’re using the Netduino with leds of 5V or 3.3V.

One final note: I used the extremely cool piranha leds. I had some difficulty understanding how to connect them, but it’s really easy. In these leds, one of the corners of the square capsule is trimmed. In the leds I bought, this corner is the cathode (equivalent to the short pin of a regular led, should be connected to the emitter in the transistor. Piranha leds are very bright, and they look very good.

So what can you do with this? For starters, you can power 4 leds using 2 pins (using transistor boolean magic). For instance, I used a NPN and a PNP transistors, and I had (on 1 Netduino pin) the same alternating effect, by turning on and off the pin. The only difference is that I connected the same pin to both resistors, and I replaced a BC547B transistor with a BC327B transistor. Check it out:

Single pin alternating effect

You can also use the super bright blue leds, which are 5V.

Playing with Netduino

6 Comments | Oct 09, 2010

I received (by chance, mostly) a Netduino last tuesday. Netduino is an open-source hardware project based on Arduino and the .Net Micro Framework. And it’s extremely cool. I’m a C# developer and I was always interested in electronics. This is an excellent tool to start to understand that incredible world. I was about to buy an Arduino when the Netduino fell on my lap, so I was really happy about it. Netduino has various disadvantages over Arduino (for starters, it’s more expensive, not all the shields for Arduino work out of the box with Netduino yet and also I’ve read that the prototypes people create in Arduino are made into products with almost no tweaking, which is not as simple with Netduino) but Netduino has an extremely cool development environment (Visual Studio), it’s developed in .Net which I’m very experienced in and you can break and inspect variables as in any other Visual Studio project.

Out of the box you get the little board, with just one button and one led. That doesn’t give you a lot to work with other than turning the led on and off with the button and the morse code “Hello World”, which was boring after a day or so. So while my maker shed super kit gets here, I tried to make the most of it.

I’m a software guy. That makes most of the samples I’ve seen look choppy. I wanted really good abstractions that I will surely a lot once I start playing with the shields. All the hardware stuff in the code I’ve seen is too exposed. So the first thing I did was just that, creating some abstractions.

The onboard led in the Netduino is mapped to an object with the following very simple line of code (everything looks simple in Netduino):

this.led = new OutputPort(Pins.ONBOARD_LED, false);

The ONBOARD_LED is the pin in the board that the led is connected to, the false means that the led will start turned off. To turn it on, you write this:

led.Write(true);

and you write this to turn it off:

led.Write(false);

And that’s about all that you can do with a led. The pin the led is connected to is a digital pin, which means that it only has On and Off states (which, as far as I could understand, means “full voltage” and “no voltage”). The abstraction for led is then:

    public class Led
    {
        private Cpu.Pin pin;
        private OutputPort led;

        private bool state;

        public Led(Cpu.Pin pin)
        {
            this.pin = pin;
            this.state = false;
            this.led = new OutputPort(pin, state);
        }

        public bool State
        {
            get
            {
                return state;
            }
            set
            {
                state = value;
                led.Write(state);
            }
        }

        public void TurnOn()
        {
            State = true;
        }

        public void TurnOff()
        {
            State = false;
        }

        public void Invert()
        {
            State = !State; 
        }

        internal void Blink(int time)
        {
            TurnOn();
            Thread.Sleep(time);
            TurnOff();
        }
    }

Notice I also added a Blink method to turn on and off the led. This is something very common, when you only have a led!!

The button is also very simple to create. You can use either an InputPort or an InterruptPort. The InputPort lets you read the state of the button, but the InterruptPort also lets you receive an event when the button is pressed or released, which is nicer (and also saves battery, since you’re not using the processor to query the button all the time). So, to create a button with the InterruptPort, you use the following line:

this.port = new InterruptPort(Pins.ONBOARD_SW1, false, Port.ResistorMode.Disabled, Port.InterruptMode.InterruptEdgeBoth);

The first parameter, as with the led, specifies the pin where the button is connected. The second and third parameters should be left as is (they have to do with the way the button is connected to the micro and to ground). The last one is really interesting.

There are five useful interrupt modes: EdgeLow, EdgeHigh, LevelLow, LevelHigh and EdgeBoth. The Level modes mean “when the button is pressed or not pressed” and they run once when the button gets to the correct state. The Edge modes mean “when the button is being pressed and released”, and IMHO are more simple to use. It’s all explained in this book. EdgeBoth means “when the button is being pressed or released”, which is what I needed to track. The event is called OnInterrupt and the header looks like this:

void

port_OnInterrupt(uint data1, uint data2, DateTime time)

Notice that this is not a normal EventArgs based .Net event. I’ll complain about that later. For now, now that data1 is the pin (in this case, Pins.ONBOARD_SW1 always), data2 is 0 when the button is pressed and 1 when it’s not and time is the time when the button was pressed. The inversion for data2 is weird, but it makes sense, I think. My abstraction for the button looks like this:

    public class Button
    {
        private InterruptPort port;
        private Cpu.Pin pin;

        public event NoParamEventHandler Pressed;
        public event NoParamEventHandler Released;

        public Button(Cpu.Pin pin)
        {
            this.pin = pin;
            this.port = new InterruptPort(pin, false, Port.ResistorMode.Disabled, Port.InterruptMode.InterruptEdgeBoth);
            this.port.OnInterrupt += new NativeEventHandler(port_OnInterrupt);
        }

        public bool IsPressed
        {
            get
            {
                return !port.Read(); 
            }
        }

        void port_OnInterrupt(uint data1, uint data2, DateTime time)
        {
            if (data2 == 0)
            {
                OnPressed();
            }
            else
            {
                OnReleased();
            }
        }

        private void OnReleased()
        {
            if (Released != null)
            {
                Released();
            }
        }

        private void OnPressed()
        {
            if (Pressed != null)
            {
                Pressed();
            }
        }
    }

With this, I only need an abstraction for the Netduino board itself. That abstraction looks like this:

    public class BaseBoard
    {
        public Button Button { get; private set; }
        public Led Led { get; private set; }

        public BaseBoard()
        {
            Button = new Button(Pins.ONBOARD_SW1);
            Led = new Led(Pins.ONBOARD_LED);
        }
    }

Easy, isn’t it? So what can I do with this? My test project included a state machine.

It works like this:

I start in a StandBy mode. If I press and hold the switch, I enter a “Recording” mode. This will go on until I do nothing for 5 seconds. After that, pressing the switch will replay what I have recorded. If I press and hold again, I enter recording mode and can change the recording. The code is extremely simple. In the Program, I simply have to create the board, attach events and state changes. There’s also an Infinite Sleep at the end of the Main method. If you leave that method, nothing else happens.

This is the code for it:

    public class Program
    {
        private static BaseBoard board;
        private static IState state;

        public static void Main()
        {
            board = new BaseBoard();
            board.Button.Pressed += OnPressedEvent;
            board.Button.Released += OnReleasedEvent;

            state = new StandByState(board, new ArrayList());
            state.StateChanged += state_StateChanged;

            Thread.Sleep(Timeout.Infinite);
        }

        static void state_StateChanged(IState newState)
        {
            state.StateChanged -= state_StateChanged;
            state = newState;
            state.StateChanged += state_StateChanged;
            state.Start();
        }

        static void OnPressedEvent()
        {
            state.OnPressed();
        }

        static void OnReleasedEvent()
        {
            state.OnReleased();
        }
    }

The first state, StandBy, will change modes to Recording on press-and-hold and to Playback on press. It will also blink twice to signal that recording is started. The only thing that’s interesting about the StandBy state is the usage of DateTime.Now. I don’t think that will ever show a real clock in Netduino (as there is no clock battery) but it does show the passage of time. The Timer also works as expected.

    class StandByState : IState
    {
        public event StateChangedEventArgs StateChanged;
        private BaseBoard board;
        private IList recording;
        private DateTime pressStart;
        private Timer timer;

        public StandByState(BaseBoard board, IList recording)
        {
            this.board = board;
            this.recording = recording;
        }

        public void OnPressed()
        {
            pressStart = DateTime.Now;
            this.timer = CreateTimer();
        }

        public void OnReleased()
        {
            this.timer.Dispose();
            if (StateChanged != null)
            {
                if (DateTime.Now.Subtract(pressStart) > new TimeSpan(0, 0, 2))
                {
                    StateChanged(new RecordingState(board));
                }
                else
                {
                    StateChanged(new PlaybackState(board, recording));
                }
            }
        }

        private void OnStartRecording(object state)
        {
            board.Led.Blink(100);
            Thread.Sleep(100);
            board.Led.Blink(100);
        }

        private Timer CreateTimer()
        {
            return new Timer(new TimerCallback(OnStartRecording), null, new TimeSpan(0, 0, 2).Seconds * 1000, Timeout.Infinite);
        }

        public void Start()
        {
        }
    }

The recording state is also very simple. I created a List for state changes, and add a TimeSpan to that list every time the button is pressed or released. I also have a running timer that will change the state I don’t press the button for 5 seconds.

   class RecordingState : IState
    {
        public event StateChangedEventArgs StateChanged;
        private BaseBoard board;
        private IList recording;
        Timer recordStopTimer;
        private DateTime pressTime;

        public RecordingState(BaseBoard board)
        {
            this.pressTime = DateTime.MinValue;
            this.recording = new ArrayList();
            this.board = board;
            this.recordStopTimer = CreateTimer();
        }

        private Timer CreateTimer()
        {
            return new Timer(new TimerCallback(OnStopRecording), null, new TimeSpan(0, 0, 5).Seconds * 1000, Timeout.Infinite);
        }

        private void BlinkBlink()
        {
            int time = 100;
            board.Led.Blink(time);
            Thread.Sleep(time);
            board.Led.Blink(time);
        }

        private void OnStopRecording(object state)
        {
            BlinkBlink();
            if (StateChanged != null)
            {
                StateChanged(new StandByState(board, recording));
            }
        }

        public void OnPressed()
        {
            this.recordStopTimer.Dispose();
            if (this.pressTime != DateTime.MinValue)
            {
                recording.Add(DateTime.Now.Subtract(pressTime));
            }
            board.Led.TurnOn();
            this.pressTime = DateTime.Now;
        }

        public void OnReleased()
        {
            board.Led.TurnOff();
            recording.Add(DateTime.Now.Subtract(pressTime));
            this.recordStopTimer = CreateTimer();
            this.pressTime = DateTime.Now;
        }

        public void Start()
        {
        }
    }

The Playback state will simply walk through the list I created while recording, and invert the led state (that is, turn it on if it’s off, turn it off if it’s on).

    class PlaybackState : IState
    {
        public event StateChangedEventArgs StateChanged;
        private BaseBoard board;
        private IList recording;

        public PlaybackState(BaseBoard board, IList recording)
        {
            this.board = board;
            this.recording = recording;
        }

        private void Play(IList recording)
        {
            foreach (TimeSpan record in recording)
            {
                board.Led.Invert();
                Thread.Sleep(record.GetTotalMilliseconds());
            }
            board.Led.TurnOff();

            if (StateChanged != null)
            {
                StateChanged(new StandByState(board, recording));
            }
        }

        public void OnPressed()
        {
        }

        public void OnReleased()
        {
        }

        public void Start()
        {
            Play(this.recording);
        }
    }

I think this exercises everything you can do with a Netduino with no extra components. It will surely get way more interesting once I have the kit some servos and a breadboard. The only thing I didn’t use was multi-threading (which I will probably try, in order to cancel the playback when I press the button).

So far, I’m loving it! There are some things I found annoying on using Visual Studio with the Micro Framework. Firs, and most important, there’s no support for either Generics or Linq. I know that this is asking too much, but I’m so used to them that I missed them. Collections in Micro Framework means “Arrays”. I found that somewhat limiting. It’s also really annoying that creating a new class using the TDD refactorings will create an invalid class (one using System.Collections.Generics and System.Linq namespaces) so you still have to go to that class to make it compile. FInally, the TimeSpan doesn’t have the Total* properties. Creating extension methods is also interesting. Just take a look:

namespace BlinkingLed
{
    public static class TimeSpanExtensions
    {
        public static int GetTotalMilliseconds(this TimeSpan span)
        {
            return span.Seconds * 1000 + span.Milliseconds;
        }
    }
}

namespace System.Runtime.CompilerServices
{
    [AttributeUsageAttribute(
      AttributeTargets.Assembly
      | AttributeTargets.Class
      | AttributeTargets.Method)]
    public sealed class ExtensionAttribute : Attribute { }
}

Finally, the emulator is useless for Netduino, but I think that you can create your own and that’s the next thing I’m looking into. Other than that, it was really, really cool.

On the spectrial

One Comment | Feb 24, 2009

Last week started the spectrial, a trial between some international movie, music and tv producers and The Pirate Bay. In case you've been living under a rock for the past few years, The Pirate Bay is one of the biggest torrent trackers in the world, run in Sweden by a number of people. The spectrial (so called by TPB members) was announced more than a year ago, and now it's here. The folks at TPB want to make it as public as possible. The producers are used to making this trials very secret and introducing all sort of semi-truths and outrageous lies. They won't be able to if too many people are watching.

First let's take something out of the way: I'm absolutely on the side of TPB in this trial. That doesn't make me an anti-copyright zealot. I think producers have a right to enforce their copyright. But this is much more than that. The trial on TPB is not about copyright, it's about technology. Many people around the world are seeing that TPB is winning, and they think this is a great news for anti-copyright movements. Let me tell you, it's not. If TPB wins, nothing will happen. No international laws will change and the ruling will probably be appealed. If TPB loses the site will not go down. It'll get moved to a place where this trial hasn't set a legal precedent (as a matter of fact, it already is). But the blow on technology will be great.

Content providers have a right to enforce their copyright, but they think they have much more than that: they think they can dictate which technology is good and which technology is bad. In this case, they think that anything related to torrents is bad. And they want to ban it. No private company with partial and focused interest should be able to do that. It's as if horse raisers many years ago when the car was first introduced could have said "cars may be a danger to the horses we sell, we should get to dictate which car is ok" and then imposing all sorts of artificial limitations on what cars can do not to make the public safe but to make their business safe.

They did it before, also: your dvd has a "region code", which is an artificial limitation on where you can use your dvd because it affects the content provider's business. And the fact that the region code is enforced is not a business contract between hardware producers and content providers, it's the law. If this were a business contract, money will exchange hands. As it is the law, hardware providers all over the world are forced to do it with no compensation. And if you try to bypass it (as you should, since you legally bought both the dvd and the dvd player) you're also a criminal (in the US at least) thanks to the DMCA. And that's not the only case. You might have heard of Spore's DRM. Limiting what a legal owner can do while not stopping piracy at all. Or the infamous betamax (Sony vs Universal) trials where content producers wanted to stop you from getting a video recorder because you could potentially pirate their content.

They have a right to enforce their copyright, they don't have the right to state that a technology capable of violating their copyright must be illegal. At some point in time, they will find out that the internet can help violate their copyright and they'll try to ban the internet. It's not like they're not trying.

What I really don’t get is why the recording industry is not seizing this opportunity. The distributors may be at risk (partially) since distributing is now close to free. But the recording industry should be benefiting from this reduced cost. And benefiting for worldwide reach without international contracts. And benefiting for millions of extra eyes. And benefiting from social networking and the new fans they can get out of that. But they aren’t. Instead, they’re treating their fans as thieves. Just watch the academy awards, where they were proud of a very big wall they put in place to avoid outside people from watching what was happening in the red carpet. Got that? You’re a fan, you love a show/director/actor/whatever. You go to the ceremony just to catch a glimpse of one of these guys, and they love you so much that they put a wall there so you can’t.

That’s what the producers think of their fans lately. There is a barrier today between established producers and their fans. The fans are trying to tear it down. The producers are building a higher wall. That just doesn’t add up. And as soon as a quality content producer with quality content comes up and tears the wall the fans will go to him. There’s no doubt about that.

If I had a say, I’ll follow two very simple rules:

1) Don’t charge for something that can be had for free, either legally or illegally. When your fans can download your music/movies/etc for free and you’re charging over $40 dollars for the same content with no added benefit, they feel cheated. You know there’s other ways to make money out of the content. You’ve been doing it on TV for as long as I’ve been alive.

2) Don’t be greedy. These other ways to make money come with an added cost: they may hinder the artistic quality of your content, turning it into more business than art. Don’t do that. You’ll loose your audience, and that’s not good business.

I don’t get why content providers are not doing this, specially TV content providers. Is it that hard to place a few ads on top of the action in Prison Break, in a non-invasive way, and charge private companies for them while knowing that the series will be seen by millions of people because it’s freely available on the internet? Is it that hard to place ads that people will actually enjoy at certain points in the show, in a non-invasive way, etc etc? I simply don’t get it.

And then there’s the issue of statistics and previewing. Content on internet is very easy to publish, and it gives a lot of good feedback. Content on TV/theaters is far more complex and it doesn’t provide good feedback. Imagine a world where the content gets to the internet first, gets “prescreened” by true fans, and then goes to the regular media where lots of people will also watch it, but you already know which content had a bigger audience and you can plan accordingly.

TV is not going away anytime soon. I know producers think it is, and that worries them. We have the WGA strike from last year to prove it. But we still have a lot of good TV years. The internet is not that accessible for the average user and it’s transmission quality is yet not on par with cable TV. We technical folks do not yet have a clue on how to make it accessible for the average user. That’s the content producers’ head start. But they are not getting it. Instead, they try to turn the internet to a TV. But the internet is not (and hopefully will never be) like the TV. The internet is like a high quality video phone. The TV allows a monologue from the producers to the viewers. The internet is a dialogue. Producers should be benefitting from this dialogue.

So in conclusion, producers may be able to win this trial (although it doesn’t look like that right now), but they won’t be defending anything. I sincerely hope TPB wins, but not because of copyright. I just want the studios to wake up, and this trial may help. The betamax trial did help. The future lives in these sharing systems and sooner or later someone will notice. And when they do, producers will tell them “there’s no way that someone that’s not us can provide a quality product”. But someone will. And when he does, producers will be dead. Hopefully they realize that this is inevitable before it happens. And then copyright will die, and then we will have our flying cars and live in the future (at least in the future of media distribution).

Setup – Custom Action Type

Add Comment | Sep 18, 2008

I was modifying old setups lately, by directly editing them with Orca. It’s not such a great experience, but I really didn’t want to go through Dark for such simple modifications.

In the process, I found that one of the worst parts of the format is decoding the Type column in the CustomAction table. Because of that, I have coded a very simple application that tells me which options are selected when I input the number in the column. I hope you find it useful, it’s uploaded here: http://www.codeproject.com/KB/install/customactiondecoder.aspx

Recycle Bin

4 Comments | Aug 31, 2008

There’s something that we developers don’t seem to get about the users, and it’s that the file system is a quite complex beast to use. It provides order, yes, but at the expense of users getting lost on where their files are. And it gets worst, as there’s no easy way to solve this. In Windows, there’s only one key that will move your files to a well known location in a single key press: the Delete key. You probably know how it works: select a few files, hit Delete, and go find the selected files in the Recycle Bin.

My mother, on noticing this, started using the Recycle Bin as the place to store her information. She will find a file after minutes or hours of looking around, hit Delete and automatically know where the file will be next time she needs it. Also, the Recycle Bin is easily accessible from the desktop. That is, until one of my brothers gets a hold of her computer, notices that the Recycle Bin hasn’t been emptied in a long time, and wipe it out. There’s no coming back from that one…

Windows tried to fix this by introducing My Documents, and it failed. Most users will simply store everything in the desktop, and the reason for that is that it’s easy. My Documents sounds childish, and My Pictures, My Videos, etc really shouldn’t be contained in it. I mean, My Pictures are not a part of My Documents. My Documents are mostly Word and PDF files. My Pictures are images from friends and family. There’s no correlation between both kinds of files. Vista does a much better job of this, with the new Document structure. This structure actually makes sense and it’s easy to access from Windows Explorer at any time. It’s ordered, and it works.

That’s why I thought of my first shell extension, and I coded it over the weekend. With this extension, pressing F12 will redirect the selected documents to where they make sense: audio files go to Music, videos go to Videos, pictures go to Pictures, the rest go to Documents. This way I cover the glitch: I have a single key press separating my disorder from order. Plus, it’s not as dangerous as storing everything in the Recycle Bin.

I coded it as a Vista-specific dll, as the directory structure from Windows XP (and earlier) doesn’t make much sense to me, and I’m going to publish source and binaries as soon as I find a good place for them (probably CodeProject). It was a great learning experience (I’m not really used to native code), and I hope people will like it when I post it.

Update: Uploaded here if it survives moderation.

Setup - Patching

4 Comments | Aug 29, 2008

In case you didn’t know, Windows XP SP2 provides a very cool update feature. It works like this: you have a main product, and the Quick Fixes / Service Packs that are applied to it. When you repair your setup for the main product, all the updates are also applied. When you remove, the cleanup is done for all. And it also looks cool on the Add/Remove Programs screen:

image

You’ve probably seen it before in Vista, and thought that that was a Microsoft only thing. I really prefer the way it looks in XP. It looks like this in Vista:

image

The good news is that you can do the same for your software, and it’s actually easy! The trick is to know the MSP format (same as MSI, but for Patches instead of Installers). MSP is a format you’ve probably never heard about, but you use it all the time. Each time you get an update from Windows Update, or a download from Microsoft KB, you are downloading MSP files (within a bootstrapper, but I’ll get to that later). Also, if you have installed any Service Pack provided by Microsoft, you’re also using MSP files. This covers both cases: KB articles are Quick Fixes, Service Packs are Service Packs.

So what’s an MSP file? It’s just the difference between 2 MSI files. For the files to be comparable, their Product Code (Id attribute in Wix’s Product element) must match. They must also have Comments in the Package element and UpgradeCodes on the Product element. The rest is pretty much free, and will count as “differences”, but you shouldn’t alter the Feature tree too much if you want a logical upgrade. Also, it’s recommended that you change the Version number (changing build for Quick Fixes, and minor for Service Packs). You should consider forcing a remove/reinstall for Major Upgrades. Changing version numbers is not mandatory, but it is recommended.

To create your MSP file, then, you need 2 MSI files:

1) The setup that your client has installed in his computer and that you want to upgrade.

2) The setup you would have provided if you could, with fixes to the previous setup.

Both MSI files must have the same name, so to keep them apart you have to place them in different directories. For the directory names, I use “installed” for the first setup, and “upgrade” for the second setup.

The cool thing about this is that you can publish an upgrade for pre-installed products in MSP format, and an install for customers that don’t have the previous version in MSI format.

To create the patch, you need both MSI files to use uncompressed files (compressed=”no” in the Wix file for both). This is not something you’ll normally do, but there is a very practical workaround: extracting both files to their own directories. To do this, you’ll do:

msiexec /a installed\myproduct.msi TARGETDIR=c:\myproduct\installed

msiexec /a upgrade\myproduct.msi TARGETDIR=c:\myproduct\upgrade

This will extract all the files in the setup and new MSI files that are not compressed in each directory. The only gotcha here is that you can’t have any feature with level 0. If the level is 0, the files will not get extracted. In this case, you can safely open the setup with Orca and change the level to 1, or whatever. This change will not render the upgrade incompatible with the previous setup. If the files are not uncompressed where they should, the upgrade will not work.

Now that you have this, all you need is a PCP file. This is a file with information on what you want to upgrade. There is a good sample of this on the Wix Tutorial (http://www.tramontana.co.hu/wix/lesson4.php#4.2), by far the best resource on Wix available. There most important element in your wix file will be UpgradeImage. It contains an UpgradeFile element where you can tell the MSP which files you need to upgrade. The MSP file will contain the delta between the two files by default (making it very small in size), but you can tell it to use the whole file if you want.

Finally, you must run the mspmsi tool. This tool is part of the MSI SDK, and it comes installed with Visual Studio 2008. You can find it at C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin. If your PCP file is called upgrade.pcp, the following line will create your MSP file:

"C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin\msimsp.exe" -s upgrade.pcp -p upgrade.msp -l log.txt

If everything worked ok, you’ll have a new msp file. If it didn’t, you’ll get an entry in the log (log.txt) with a very cryptic error. Most common reasons for an MSP not working are:

1) Putting the wrong path for the MSI files, or not having run msiexec /a. In this case, you’ll actually get an error that makes sense.

2) Not having Comments on either MSI file, or having an apostrophe within the product name or the comments. This is illegal, and should be modified.

3) Non-matching signatures in the MSI files (that is, different Product Codes).

4) MSI files with non-matching file names.

If it worked, you can now open the msp file in Orca to see if the information is correct. To do that, you first open the installed MSI, and then go to Transform –> View Patch and open the MSP file. You’ll get something like this:

image

Where the green line to the left of the table names means that there are differences between the original and the patch, and the square green box means that the table is new. The Patch file in the screenshot is the most important difference. When the patch run, a PatchFiles action will be run in the Execute Sequence. This action will apply patches for every file in the Patch table. If it is empty, the setup will run but your files will not be patched. When the files in the UpgradeFile elements are not found in the directories where you have uncompressed your setup (with msiexec /a), the table will be empty.

And now you can run the setup! Simply double click the upgrade, and it will be handled by msiexec. If the previous version of the product is installed, you’ll see the Change/Repair/Remove dialog (if available). Clicking on Repair will run your patch. Notice that the setup is running all the actions that a Repair will normally run. There is a PATCH property you can use to know when the setup is being patched, and add or remove custom actions from the patch.

After you install the patch, Repair on any patch or the original MSI will result in your patch being applied. Remove in the patch will only remove the patch (and dependent patches if any). Remove in the MSI will remove all.

The Change/Repair/Remove interface may not be clean enough for your needs, though. You may need the patch to simply show a progress bar and be done with it. You can get this behavior by running:

msiexec /p upgrade.msp /passive

There is a tool called IExpress that’s used to create bootstraps for your setups. If you run IExpress from the command prompt, you’ll see it.The tool is pretty self explanatory. The only thing that you need to know is that you have to input the line I just mentioned in the “Install Program to Launch” step. When IExpress is finished, you’ll have an exe file that will get the correct behavior for you.

So go ahead, and happy patching!

If you need more information, or you want a second opinion, there’s a step-by-step here: http://trentm.com/blog/archives/2007/05/29/wix-and-msp/

You can also use the Wix Tutorial: http://www.tramontana.co.hu/wix

Finally, there’s more info (although somewhat misleading) of IExpress here: http://www.microsoft.com/technet/prodtechnol/ie/ieak/techinfo/deploy/60/en/iexpress.mspx?mfr=true

Setup - Elevation

2 Comments | Aug 15, 2008

It is very common on Vista (with UAC activated) to have a setup run through with no error, but to end up not installing anything. This happens when you try to write to protected resources (such as some parts of the file system, registry, etc). It’s easy to get past this on custom actions (using impersonate=”no” and Execute=”deferred”), but finding out how to do it on the “standard” components is not that easy. Luckily, it’s a one-liner. Just add the InstalledPrivileges attribute to your package, like this:

<Package Id='????????-????-????-????-????????????' Description='My Software' Manufacturer='MyCompany' InstallerVersion='250' InstallPrivileges='elevated'></Package>

And now your package will ask for elevation as soon as the execute sequence starts, resulting in all the actions/components being installed with elevated privileges.

Custom Action – Copying a Property to another Property

Add Comment | Aug 06, 2008

Custom Action type 50 has a very useful behavior. This custom action copies a value to a property. The cool thing about it is that you can copy any formatted string. That is, if you use [], you can get the value of another property. and you can also use environment variables (with a % prefix), file paths (with a # prefix) and directory paths (with a $ prefix). So, to copy a property to another property, simply do this:

<CustomAction Id="CopyProperty" Property="TargetProperty" Value="[SourceProperty]" />

And then you’ll have two properties with the same value!

Custom Action – Status Bar

Add Comment | Aug 06, 2008

If you’ve made your own setup using Wix and Custom Actions, you probably got to the end of your setup to see the progress bar still running with an empty status message. It’s specially notorious if your action takes a long time to run. Gladly, there’s an easy solution to that. Simply open your UI.wxs (or add a <UI> tag in your Product, or in a Fragment) and add the following line:

<ProgressText Action="MyAction">Configuring my Custom Action</ProgressText>

Where MyAction is the Id of your action and that’s it, you now have a proper message on long-running custom actions.

Product Keys and Setup – Custom Action

2 Comments | Jul 27, 2008

The question from my previous post was: how do I validate a Product Key from a Wix Setup? The answer has many components.

First, you have to create a dialog for your product key in Wix. This dialog should have some sort of formatted input (say, a repetition of six 4-letter groups). It should also have some sort of validation.

Next, you need to have a custom action that will validate the key. It’s very important for this custom action not to be obvious to crack. Wix is very easy to modify as is, if you’re going to use the same code to validate the key in your product, let them bypass the control in the setup; it will be much harder to bypass in the product.

So first things first, let’s start with Wix. It’s actually very easy.

If you’re using WixMondo, you have most of the work cut out for you. You only need to add a new dialog to the dialogs directory. This dialog should include a MakedEdit control, which is where your key will be displayed.

The mask for the MaskedEdit is somewhat complex, and the documentation isn’t good. But for this exercise, the result is actually pretty simple. A mask of & will recognize any single character or number, and the – will display a separator. My mask in this example is a property named PIDTemplate (which is a standard variable name for the product key’s mask) declared like this:

  <Property Id="PIDTemplate">
    <![CDATA[&&&&-&&&&-&&&&-&&&&-&&&&-&&&&]]>
  </Property>
The key is associated to the Control via the Text property, like this:

    <Control Id="Key" Type="MaskedEdit" Height="15" Width="330" X="32" Y="148" Text="[PIDTemplate]"  Property="KEY" />
I think that you can paste the key directly into the attribute instead of using the property, but that really didn’t work for me.

Finally, you have to provide validation via the DoAction event in the Publish section for the Next button:

<Control Id="Next" Type="PushButton" X="236" Y="243" Width="56" Height="17"     Default="yes" Text="$(loc.WixUINext)">
  <Publish Event="DoAction" Value="MSIKey">1</Publish>
  <Publish Event="NewDialog" Value="[WixUI_ProdKeyDlg_Next]">VALIDATEKEY =  "1"</Publish>
  <Publish Event="SpawnDialog" Value="BadKeyDlg"><![CDATA[VALIDATEKEY <> "1"]]></Publish>
</Control>
Notice that the events will run in order. That is, your MSIKey will run first (setting the VALIDATEKEY property in the process), followed by the NewDialog if VALIDATEKEY equals 1 and the SpawnDialog if the VALIDATEKEY doesn’t. NewDialog will replace your current step with the following step (defined in UI.wxs) and SpawnDialog will show a new smaller dialog with an error, that you must also define.

The full code for the dialog will then be (sorry for the formatting, the whole point is to give you a good copy paste point):

<Include>
  <Property Id="PIDTemplate">
    <![CDATA[&&&&-&&&&-&&&&-&&&&-&&&&-&&&&]]>
  </Property>

  <Dialog Id="ProductKeyDlg" Width="370" Height="270" Title="$(loc.ProdKeyDlg_Title)">
    <Control Id="Key" Type="MaskedEdit" Height="15" Width="330" X="32" Y="148" 
          Text="[PIDTemplate]"  Property="KEY" />
    <Control Id="Back" Type="PushButton" X="180" Y="243" Width="56" Height="17" 
          Text="$(loc.WixUIBack)">
       <Publish Event="NewDialog" Value="[WixUI_ProdKeyDlg_Back]">1</Publish>
    </Control>
    <Control Id="Next" Type="PushButton" X="236" Y="243" Width="56" Height="17" Default="yes" 
           Text="$(loc.WixUINext)">
       <Publish Event="DoAction" Value="ValidateKey">1</Publish>
       <Publish Event="NewDialog" Value="[WixUI_ProdKeyDlg_Next]">VALIDATEKEY = "1"</Publish>
       <Publish Event="SpawnDialog" Value="BadKeyDlg"><![CDATA[VALIDATEKEY <> "1"]]></Publish>
    </Control>
    <Control Id="Cancel" Type="PushButton" X="304" Y="243" Width="56" Height="17" Cancel="yes" 
           Text="$(loc.WixUICancel)">
       <Publish Event="SpawnDialog" Value="CancelDlg">1</Publish>
    </Control>
    <Control Id="BannerBitmap" Type="Bitmap" X="0" Y="0" Width="370" Height="44" TabSkip="no" 
           Text="$(loc.ProdKeyDlgBannerBitmap)" />
    <Control Id="BannerLine" Type="Line" X="0" Y="44" Width="370" Height="0" />
    <Control Id="BottomLine" Type="Line" X="0" Y="234" Width="370" Height="0" />
    <Control Id="Description" Type="Text" X="25" Y="23" Width="280" Height="15" Transparent="yes" 
           NoPrefix="yes" Text="$(loc.ProdKeyDlgDescription)" />
    <Control Id="Title" Type="Text" X="15" Y="6" Width="200" Height="15" Transparent="yes" 
           NoPrefix="yes" Text="$(loc.ProdKeyDlgTitle)" />
    <Control Id="EnterKey" Type="Text" Height="20" Width="154" X="25" Y="116" 
           Text="$(loc.ProdKeyDlgEnterKey)" />
  </Dialog>
</Include>

I will spare you the code for the error dialog. If you need one, just copy Cancel.wxi or OutOfDisk.wxi and change the name and description. Add it in the includes section of UI.wxs, and you’re good to go.

Next comes the Custom Action, which is actually the hardest part. In MSI, you can create a Custom Action in several ways. The most common for this kind of thing are JScript/VBScript, Managed (as in, C#) and Native. My reasons for using native, following:

1. Nobody in their right mind should right a piece of JScript code and think that it’s safe. Period. You can easily extract the JScript from the MSI, and then your product will not be safe because you have given any potential attacker your key validation code. Also, JScript is somewhat harder to debug than the other two options. JScript can’t be used from your product’s code (most of the time), so you’ll have to duplicate your logic and your code to call it if needed.

2. Managed is a little harder to exploit, but just a little. The real drawback here is that you can’t really use Managed code in the DoAction event of a Control. If you really want to, you can refer to a post in Pablo’s blog that tells you how to do it. It’s a very hard and very manual process you’ll have to do every time you compile your assembly which basically fools the MSI executable into thinking that your code is native when in fact it isn’t. Managed can be used from you code easily from your product if your product is also written in the same managed code. I don’t think you can use Java for a custom action in MSI, but I really didn’t check into it.

3. Native is the hardest to exploit, but also the hardest to write, especially if you’re not used to it. Also, it’s the recommended choice for any custom action run from an MSI. Finally, a native dll may be imported in almost any language that you are using for your product.

There’s also another option you have available for the custom action: wether the dll containing it will be stored in binary form somewhere in the database, or if you want to store it in the file system before running it. I’m going to show how to use it from binary, but you’ll usually store it in the disk if you want to use it from your product too. Either way, Wix configuration is almost the same:

<Binary Id="ProductKeyCode" SourceFile="Binary\ProductKeyCode.dll" />
<CustomAction Id="MSIKey" 
     BinaryKey="ProductKeyCode" DllEntry="MSIValidateKey" />

The only difference between this and using the deployed version is that you will have a FileKey instead of the BinaryKey, and the dll as part of your regular directory structure.

And finally for the complex part, here’s how you create the dll. I’m not a native developer by trade, so the following may have huge conceptual errors. Please forgive me for them and help me correct them using the comments.

First, you have to create a Win32 application in Visual Studio, and select the type as Static Library in the wizard. Don’t include MFC or Precompiled Headers. The wizard will create a .cpp and a .h file for you to edit. It will also include a set of std files.

Open the stdafx.h file, and add the following:

#include <msi.h>
#include <msiquery.h>
#include <stdlib.h>
#pragma comment(lib, "msi.lib")

#include <comutil.h>
#include <string>
#include <tchar.h>

The msi.h and msi.lib are important, the others are being used only for this sample. You may not have the msi.h, msiquery.h or msi.lib available in your computer. If you don’t, you’ll need to install the Windows Installer SDK to get them. Also, you may need to add the folder where the msi files are to your includes and libs directories, or copy the files to the actual include and lib directories.

The actual code (in your cpp file) should look like this:

#include "stdafx.h"
#include "ProductKeyCode.h"

int __stdcall GetItemFromKey(TCHAR* szValueBuf, int iPos)
{
    std::basic_string<TCHAR> str = szValueBuf;
    wchar_t ptrKey [5] = TEXT("****");
    str.copy(ptrKey, 4, (iPos - 1) * 5);

    int iRet = wcstol(ptrKey, (wchar_t**)NULL, 16);

    str.~basic_string();

    return iRet;
}

BOOL __stdcall KeyValidationMain(TCHAR* szValueBuf)
{
    int key0 = GetNumberFromKey(szValueBuf, 1);
    int key1 = GetNumberFromKey(szValueBuf, 2);
    int key2 = GetNumberFromKey(szValueBuf, 3);
    int key3 = GetNumberFromKey(szValueBuf, 4);
    int key4 = GetNumberFromKey(szValueBuf, 5);
    int key5 = GetNumberFromKey(szValueBuf, 6);
        
// Your validation goes here. 
    return TRUE;
}

UINT __stdcall MSIKeyValidation(MSIHANDLE hInstall)
{
//    MessageBox(NULL, TEXT("Debug"), TEXT("Debug"), 0);
    TCHAR* szValueBuf = NULL;
    DWORD cchValueBuf = 0;
    UINT uiStat =  MsiGetProperty(hInstall, TEXT("KEY"), TEXT(""),
                                     &cchValueBuf);
    if (ERROR_MORE_DATA == uiStat)
    {
        ++cchValueBuf; // does not include terminating null, so add 1
        szValueBuf = new TCHAR[cchValueBuf];
        if (szValueBuf)
        {
            uiStat = MsiGetProperty(hInstall, TEXT("KEY"), szValueBuf
                    , &cchValueBuf);
        }
    }
    if (ERROR_SUCCESS != uiStat)
    {
        if (szValueBuf != NULL) 
        delete [] szValueBuf;
        return ERROR_INSTALL_FAILURE;
    }

    if (ValidationKeyMain(szValueBuf) == TRUE)
    {
        MsiSetProperty(hInstall, TEXT("VALIDATEKEY"), TEXT("1"));
    }
    else
    {
        MsiSetProperty(hInstall, TEXT("VALIDATEKEY"), TEXT("0"));
    }

    delete [] szValueBuf;    

    return ERROR_SUCCESS;
}

There are two exported methods here: MsiKeyValidation and KeyValidationMain. GetItemFromKey is a private helper method that may be declared elsewhere. Notice that the actual validation occurs in KeyValidationMain. This method will run your validation algorithm (a cryptographic thing, mostly) and return TRUE if the key is valid or FALSE otherwise. My convention is that, in case of error, I return false. MsiKeyValidation is a simple wrapper to integrate with MSI.

The commented MessageBox at the beginning of MSIKeyValidation is important, I’ll get back to that in a second. MSIKeyValidation will get the KEY property from the Msi database. It will always look the same: “    -    -    -    -    -    “, with whatever the user writes in the spaces or spaces if the users doesn’t write anything. This makes it very safe to cut in the GetItemFromKey method. In this last method, I return a base 16 number for each part of the key (I use hex in this sample), but you’ll generally use it directly as text.

Finally, to make the exported files visible, you have to include a .def file, with the following contents:

LIBRARY    "ProductKeyCode"
EXPORTS
    MSIKeyValidation
    KeyValidationMain

This library should compile as is. If you open it with depends, you should be able to see the functions in the exports section. A setup constructed this way will work on most computers, but I found out that sometimes, it doesn’t work. If you’re finding that your dll isn’t running (and you go to the %temp% directory, and you don’t see it appearing there, and you can’t attach to it), then go to Project Properties, find C/C++ –> Code Generation and select Multi-Threaded Debug in the Runtime Library option. I really don’t know why, but it will fix it.

With this done, you’re ready to deploy with a product key. If you want to debug the dll, you’ll have to reenable the message box. Run the setup and wait for the message box to appear (it will appear when you click Next on the Product Key dialog). At this point, attach Visual Studio to the newest msiexec process, and you should be able to debug. The executable for Msi opens a new process for each Custom Action. In any case (script, managed or native) the correct way to attach a debugger is the same.

As an added bonus, if you want to call your validation from managed code, you should write the following code:

[DllImport("ProductKeyCode.dll", CallingConvention = 
CallingConvention.StdCall, EntryPoint="KeyValidationMain")]
static extern bool KeyValidationMain(
[MarshalAs(UnmanagedType.LPTStr)]string szKey);

And that’s it! Now you can access the same algorithm from managed code.

Product Keys and Setup – Random Thoughts

3 Comments | Jul 26, 2008

I was commissioned recently on making a Product Key system for a shrink wrap piece of software. And my first thought was: there must be some sort of best practice or pattern language for this. I mean, many people sell shrink wrap software every day, and inventing the procedure each time is just to costly and error probe. Turns out, I was wrong. There is no pattern or best practice for product keys (at least none that I could find in the internet, forums or books). It makes sense, though, for several reasons. First, the market for enterprise software is way bigger than the market for shrink wrap, and there’s where all the advise is directed to. Next, because product keys are something best kept secret. As we know, security by obscurity was never a great idea, so let me share what I have come up with.

Before I start, let me tell you Joel has it right, twice: he says that product keys aren’t a good measure of security (at least I think it was him, I can’t find the link right now). Crackers will crack them anyway sooner or later. Or they can end up in public or not so public forums in a very short time. Next, he says you should raise up your prices and he’s also right. Both pieces of advise go together: you don’t put a public key in place to make your software impossible to copy. You put it in there to clearly state that making such a copy is illegal. Normal customers, users, will get a pirated copy of your software if they want to. It’s not hard and as of today it's not dangerous. But the corporations can’t do that, because they have codes of ethics to attend to. Also, their bottom line is not as affected by you as a regular user’s is. So my comment on this is: use a product key, so a customer will get your software from wherever he finds it, adopt it, and tell his boss about it so he can buy it. You may even get someone so into your software that he’ll buy it from you right after getting an illegal copy of it.

One very valid solution is to avoid the whole product key affair, and publish a free version of your software with a reduced set of features in it, and a full version on purchase. Of course, the rules of the game will remain the same. Pirate copies of your software will be unprotected. Remember: seeing your software in the pirate circles means your software is successful. If a cracker took the time to work on it, it probably means it’s worth it. This fact will not get you any money now, but it sure will make you money in the long run.

My general advice is:

1. Make your key easy to code. Not as easy as to make your key laughing stock from the cracking community, not as hard as to lose valuable time (your time) over it. Crackers will probably bypass your code instead of trying to solve your algorithm if it’s too hard.

2. Always use some sort of non-managed code for the key algorithm. Managed is just to easy to read, even when obfuscated. I mean, this guys debug your code in assembly to crack it, scrambled names will not be enough to stop them. Also, if you’re a Managed developer, remember that a signed assembly cannot be tampered with, but it can be read.

3. Validating your key in the setup is cool for usability, but it has nothing to do with security. MSI is a simple database and it can be very easily modified. Most software providers actually validate the key in an exe before the msi even starts up (that’s one of the reason that most software ships with setup.exe and setup.msi). I find that doing that is not really necessary if you do 4.

4. Always validate your key on startup of your product. If the key is invalid, you can load with a reduced set of features, or not load at all.

5. If you want to provide a time bomb (that is, trial software that will eventually expire), hide the expiration date (or something that you can convert to the expiration date) somewhere in the key.

6. Eliminating keys you find on the internet is an impossible race. There’s many more of them than there are of you. New keys will resurface within hours.

A good product key is like a good encryption algorithm. In this context, that means two things:

7. If you change a single bit of your product key, the rest of the key should change to accommodate your checksum. All the “digits” must change.

8. There should be a minimum (say, 10%) of valid product keys per bit. That is, if your product key is 32 bits (it won’t be), only 2 or 3 keys should be valid.

I created the key on a very simple algorithm I will not disclose, using Wix and a C++ dll. I’m planning on giving the details of the procedure on the next few posts.

Matching Braces with Regex

4 Comments | Jul 24, 2008

3 years ago, I wrote this post: http://regexlib.com/REDetails.aspx?regexp_id=1135.

It contained a .Net regular expression that would match the highest level of matching braces, something that is needed more often than not when parsing languages. It will also return null if no matching braces are found. Finally, as an added bonus, it will ignore braces that are escaped with a backslash.

Soon after I posted it, someone commented on it being impossible, because Finite Automatas can’t count. I didn’t want to fight it (and I didn’t know better), so I simply made a wrong statement that would divert the discussion. I simply knew that it worked back then. I’m wiser now, and I know the answer.

FAs are used to parse regular languages and as such, they can’t count. That’s absolutely true. That means that you can’t keep track of what you’ve seen by using a FA. So, if you need to match a symmetrical phrase (aabbccbbaa), you can’t. That’s what Push-down automatas are used for.

Problem is, .Net regular expressions don’t work as a regular language. When you start using backtracking and lookahead, you’re automatically out of the regular language domain. And then, Finite Automatas can’t be used anymore.

So, if you need to count, you can always use a backtrack. You can use them to search for a palindrome (http://aspn.activestate.com/ASPN/Cookbook/Rx/Recipe/326097) and you can use them to search for matching braces.

As an added bonus, here’s how you do different braces with the regex:

Curly ({}): (?<!\\)\{(\\\{|\\\}|[^\{\}]|(?<!\\)\{.*(?<!\\)\})*(?<!\\)\}

Parenthesis (()): (?<!\\)\((\\\(|\\\)|[^\(\)]|(?<!\\)\(.*(?<!\\)\))*(?<!\\)\)

Square ([]): (?<!\\)\[(\\\[|\\\]|[^\[\]]|(?<!\\)\[.*(?<!\\)\])*(?<!\\)\]

Chevrons (<>): (?<!\\)\<(\\\<|\\\>|[^\<\>]|(?<!\\)\<.*(?<!\\)\>)*(?<!\\)\>

Hope you find it useful!