Author Archives: aenikata

Technical tests, useful and not so useful

Having worked at a number of places, I have, of course, done my share of technical tests. I know the form for this, and am perfectly comfortable writing code that has comprehensive unit tests, full comments, documentation, consideration of SOLID principles, which is tested against the examples, etc. I have an almost perfect track record when it comes to these, because generally they ask for a reasonably well-defined piece of logic that’s not excessive in scope and which doesn’t expect any niche knowledge that you can’t easily look up. Mostly they’re good tests, because they establish that you can write working code that someone else could work on. You can demonstrate an ability to work in a style, and the tests can be done in a candidate’s own time to minimise interfering with existing commitments.

The one time I failed a test was having already had a strong track record, and therefore came as something of a surprise. The company apparently felt that all the above elements weren’t sufficient, despite stating explicitly that there was no need to ‘gold plate’ the test. The issue seems to have been that they also expected a full algorithm implementation on top of all that – from the technical test options, either a full Constraint Optimisation algorithm for meeting planning, a Graph Exploration algorithm for route planning, or a Natural Language Parser for interpreting free text input. From an unpaid piece of work that will just be thrown away afterwards.

I’m sorry, but no. If you pay me to implement a meeting planner I’ll do constraint optimisation (although I’ll probably use the available libraries for the job since it’s likely to be more efficient than my reinventing the wheel). Similarly if you want to work on a commercial project creating a bot or similar tool and therefore need to understand free-form natural language, then I’ll work on a full NLP system, although again I’d normally use libraries since even 60′s-era expert systems are non-trivial amounts of work. I should also point out that I’d equally happily work on an Open Source project that works towards these kinds of system. It’s not about being paid directly, but about my time being worth something. Working on a project that someone may use creates something of value from my time, but implementing an extensive but throw-away technical test doesn’t.

I would ask the question about what a company like this hopes to achieve by creating such a technical test. The first impression that it sets is that they place little value on the time of the potential employee, because it demands unpaid effort so freely. This can’t create a good impression about the company because any company which is willing to make such demands at this point seems likely to make similar demands of its employees, which doesn’t imply a good work-life balance.

The second is that they do not care about being clear in their requirements. One reason for relatively simple technical tests is that it is easier to be very clear about what is and isn’t expected. A complex with with a caveat not to gold plate leaves a candidate guessing how much to comment, document, refactor and restructure for SOLID principles, validate against bad inputs, handle additional use cases, unit test, plan for extensibility, etc. Most candidates will guess wrongly, but that’s not their failing, it’s the company’s, because it has failed to communicate its expectations then blamed the candidate when they haven’t magically known. This isn’t a project where the goals are not clear, so it’s relatively easy to be very detailed in your specifications. By all means expect Agile practices like test-driven development, but because there aren’t daily stand-ups, stake-holder participation, etc, some communication has to revert back to a more traditional specification document approach. I’ll push for Agile practices in the workplace, but I’ll equally recognise where they won’t work, and will adapt my practices accordingly, and a company should, too.

A further impression is that the company has not read recent articles about changing hiring practices at companies like Google, who have moved away from tests like these that demand specific algorithm knowledge. They did this because they concluded that these tests do well at boosting the ego of the staff involved in setting the test but perform poorly at identifying good candidates to hire. The reason for this is simple – a recent graduate is liable to have spent some time looking at algorithms and may have recent exposure to these, so they will tend to do reasonably well. By contrast, someone with decades of experience will have any algorithm work from their academic studies a dim and distant memory, long since replaced with real-world knowledge about system security, data protection, CSS and JavaScript, data warehousing, data analysis, database optimisation and indexing, message queues, multi-threading, numerous libraries, and so on. Why set a test that may well inadvertently favour the wrong candidates?

To be clear, I did wonder whether a more advanced implementation was expected. The test was much longer than any other I had received in the past and included instructions not to ‘gold plate’, so I tried to balance how complex a system to implement with that instruction. I decided that it was most appropriate to focus on handling the defined cases and adding appropriate tests, that a full complexity algorithm would not be required for such a test, and apparently this was wrong. I was left guessing on scope, not on how to implement logic. so their conclusion is I don’t have sufficient knowledge when what I didn’t know was how much was expected rather than how to implement what they expected. It’s the wrong conclusion resulting from asking the question in the wrong way. I have implemented more complex Natural Language Processing than I implemented for the test (although my language for that was Prolog as a tool well suited to the task, rather than C#). I’ve also done some Graph Exploration algorithms (if you’ve seen Project Euler you’ll know that various of these questions cannot be solved without some similar consideration of how to avoid brute force calculations, and I’ve looked at route planning for a robot vehicle, implementing code in Matlab and Python previously). Constraint Optimisation I admittedly haven’t done much of, but I know that naive algorithms can hit severe performance issues and would generally prefer to use libraries that are likely to prove much more efficient than any first-cut algorithm I’d implement.

The strangest bit about the whole test was the requirement not to use 3rd party libraries. If you want to work efficiently, don’t reinvent the wheel. I put more effort into learning the principles then appropriate tools than in writing toy imitations of what’s already available. I’ll use R or Pandas (in Python) or Accord.NET (in C#) for machine learning , despite having implemented a basic neural network and other algorithms in Octave, because the tools are there and more performance than the code i’d write. This requirement will favour those who reinvent the wheel over those who don’t. I’m valuable because I know the right tools to use. For example, I haven’t implemented quick sort and bubble sort and so on, because working in C# there’s no need to. After implementing a Neural Network (for a course) I’ve not used that code since, because I get better performance from the libraries in R and Matlab. There have been many articles about ‘NIH’ (Not Invented Here) syndrome, where significant resources are wasted implementing something that’s buggier and slower than 3rd party libraries that do the same job. I’ll ALWAYS call for identifying a complete toolbox first. Occasionally you’ll need to write lower level tools or libraries. However, in the same way as you wouldn’t use Assembly to write an e-commerce site you’re also mostly better off understanding the basic principles of an algorithm (to understand the limitations and requirements) then using an existing, well-tested implementation where one already exists. I’d rather hire someone who can identify the best libraries to use than someone who can write a pale imitation of them.

I have a job. Between that and commuting that’s around 65 hours a week, currently. I have a family, and since I’m away during the week they expect my attention when I’m around at the weekend. I also need to sleep. Ask too much of my time unpaid and I’ll just say no. It’s not just about placing a value on my time, but if I push too hard I’ll get tired. I’m not going to risk doing a bad job for an existing client by being tired, because that would not be professional behaviour. Perhaps those companies asking candidates to implement extensive technical test pieces should consider what kind of candidate would put in that much time. Candidates between roles will do everything they can, but those who are sufficiently in demand to be currently engaged should be prioritising their current commitments over pursuing their next step. I don’t want to be the kind of person that puts those priorities the other way around, and I’d suggest that no company wants that kind of person either.

At the end of the day I’d say companies should consider their principles and the principles of the people they want to hire. Treat your candidates like their time is valuable and don’t waste it. Expect them to treat their existing client/employer as their top priority. This one wasn’t about the money (it would have been a pay cut), but it upset me because the company talked about social justice. They talked a good talk but didn’t walk the walk.

So what are the solutions? Well, for working out whether you want to work with me, drop me an email, and I’ll happily send over example technical tests I’ve done in the past. I’m not afraid they’ll make me look bad, and I’d rather not keep doing random other throw-away developments that create no value. Why not ask potential candidates if they have any examples of code they can send for evaluation before you ask for them to implement something throw-away. Alternatively, if you want to specify something new, how about thinking about things in a different way? Why not hire them for a day to work on something small but real? Alternatively, point them at an Open Source project and ask them to do some work on it, or ask them to identify code they’ve contributed to an Open Source project? If you work with charities, maybe get something implemented for the charity. There’s lots of ways to ensure that valuable time is not wasted on creating something with no actual value while still getting the information you need. Most of these suggestions wouldn’t tend to demand specific algorithm knowledge and avoid some of the pitfalls of the complex algorithm implementation technical test.

Completion of the basic elements for a budget telepresence robot

It’s taken a bit longer than expected, but the software and hardware elements are now all present for the telepresence robot. The basic concept is a tablet with a mount attached to a base that has an ESP8266 wifi microcontroller and motor board that controls 2 motors. There’s various options for powering the tablet and the base, but a good USB battery pack can run it for a reasonable time.

The initial plan was to use a Doit.am wifi car as the base, which provides the microcontroller, motor board and motors with a mount. If you want to try mounting a phone for a mini option then the base could be fine, but it turned out it wasn’t large enough for mounting a tablet at a reasonable height off the floor – it wasn’t stable. Additionally, tests with the included motors found that they struggled to move the combined weight of the components. 

As a result, I evaluated other low-cost motors to see which looked most suitable for the base. I settled on the N20 micro gear motors, which provide a suitable level of torque to cope with the weight of the components. Cost-wise these are comparable with the motors in the doit car, but combined with the smaller included wheels results in slower, more reliable movement. 

The tablet is just attached in the tablet mount on the tripod, which is trivial. To provide a base for the motors and to hold the motor control components, I used a 48cm round plant saucer from Wilko. The tripod legs needed flexing slightly to fit inside at the height I wanted, but the height and width are somewhat adjustable. I decided to have the tripod legs at the standard spread for stability, and this seems to be sufficiently stable in use so far. While it’s not hard for someone to knock over a tablet mounted like this the addition of a base with the additional weight of a battery pack helps make it more stable.

To mount the legs in the saucer, I just drilled 2 holes next to each leg and cable-tied the legs to the saucer. The tripod has clips for adjusting the leg extension, so once the cable ties go over these they hold the tripod in place reasonably securely. Naturally, this could be hot-glued in place for a more permanent solution or a mounting block 3d-printed, perhaps, but cable ties provide a quick, simple solution that works well enough for the initial build.

Mounting the wheels and motors were trickier. Ideally you probably want to mark out rectangles for the wheels and then use a rotary power tool to cut out the sections. I didn’t have one to hand, so resorted to drilling out sufficient holes and then cutting out the remaining plastic – it looks a bit rough, but so long as you have enough space for the wheels to go through and move freely, it’s fine. With the doit wheels, I was thinking of mounting the motor on the top and having the wheels pass through to minimise the ground clearance, but with the N20 motors the smaller wheels wouldn’t give enough ground clearance, so these seem best mounted on the bottom. Again, for an initial solution, I drilled 4 holes around each motor and cable-tied the motor to the saucer (note, the motors will probably need to have had wires soldered on before you mount them, otherwise it will be fiddly). You’ll need a castor wheel at the back, and with 4 spare screws I drilled some holes and bolted this onto the saucer, although in a pinch hot glue or more cable ties could probably suffice.

The wiring up is easy – there’s a positive and negative for each motor. If you’re using just USB power, you don’t need to wire in VM or GND on the motor board. If you want to provide more power to the motors for more speed, wire the battery pack into VM and GND – then the push-switch on the motor board will provide on-off control. If you’re powering the board from a separate battery pack, set the jumper accordingly – a 7.4V battery pack can run the microcontroller as well, although you possibly want that battery pack to power just the motors.

The USB battery pack is used to power the microcontroller and the tablet and optionally the motors – they’ll run slower at 5V, but it should still run. Depending on the drain from the tablet, the battery pack may provide a substantial runtime, although using the motors driven from the pack will quickly reduce this. The optional component here is Qi wireless charging. A Qi receiver would be plugged into the microusb charging socket on the battery pack and mounted at the back of the unit – most likely at the edge of the saucer. The Qi charger would need to be mounted somewhere suitable for it to come into contact – at a similar height on the wall. The idea is that if you carefully back into your designated location the receiver can get close enough to charge the battery and ensure when you do move off it has a full charge, With minimal movement, it should be possible for the battery pack to run the robot for a week without charging, though. 

That basically completes the hardware build. As for the software, the tablet just needs to run Skype. I opted for a Kindle Fire HD because the camera is reasonable for the price. I discovered that Fire OS isn’t happy to leave wifi on while on standby, so I ended up having to install Cyanogenmod on it in order to control that – this is essential as otherwise when you try to dial into the tablet you won’t be able to connect. There’s still been some issues – it has sometimes gone on standby or crashed, so if you can get a similar qualify native Android tablet like a Samsung Tab, it may be a more stress-tree option. The only thing that needs to be installed on here is Skype. Create a new account for the robot, and set Skype to auto-answer. Make sure that only known contacts are allowed to call (you don’t want anyone to call and have auto-answer) and add your existing Skype account as a contact so you can phone in.

I did consider open source alternatives such as Linphone, and it will probably be interesting to look at an integrated solution that provides a telepresence interface with both video and motor controls in the one window, but that’s a project for later. The stock Linphone build doesn’t have auto-answer as an option, so it’s not really suitable as-is, although it’s apparently available as an option if you’re doing a custom build. 

For the motor base, the default software for the NodeMCU dedicated board allows you to install the Doit wifi car app and control it by connecting to its access point. This would allow you to control the robot within a moderate range – up to 100m or so, depending on interference, etc. However, that’s not really what you want for a telepresence robot – we want to control it over the internet. There is supposed to be a remote version which connects to a server and allows the board to be controlled over the internet. However, the documentation is so bad that it’s entirely unclear where you’re supposed to do in order to control the car. As a result, I ended up taking the DoitCarControl.lua script from the Doit site and removing the code to set up an access point and renamed it DoItCarControlSTA.lua, modified the sta.lua script to connect to my local wifi network, and uploaded the remote (STA) init.lua script, modified sta.lua script and new DoItCarControlSTA.lua script onto the board. This was loaded using ESPlorer. I found I had to get the NodeMCU flasher to reinstall the lua interpreter, needed to locate a driver so the board showed up as a COM port when plugged into the PC, and sometimes the scripts didn’t upload properly – sometimes I got an error about being out of memory and the board needed resetting using the Reset button on the board. 

Having put these scripts on the board, it connects to the local network and picks up an IP address. Going onto my router I opened up port 9003 and set it to forward to this IP address – you’ll ideally want to configure your DHCP so it assigns the same IP address each time, or pick a fixed IP address that’s outside of the DHCP range and set this in the sta.lua script. Now it should be possible to connect to the telepresence robot from anywhere.

The remaining question is how to control it. The doit app is no more happy controlling this version as it is controlling the normal STA app. Instead, go on Google Play and search for Wifi TCP/UDP Controller. This provides a configurable page of buttons which can send TCP or UDP messages. If you open up the DoItCarControlSTA.lua script you’ve made you’ll see the values that correspond to forwards, backwards, left, right, stop, faster/slower left motor and faster/slower right motor. Set up the buttons in the app so that they send the appropriate values. Enter your IP address or domain name for what to connect to, and make sure the port is set to 9003 (unless you changed this in the lua script). 

Having assembled the hardware and installed the modified software on the NodeMCU board, Skype on the tablet, Wifi TCP/UDP Controller on your phone and Skype on another device to talk from and configured the router to pass through requests to the given port, it should be possible to call into the tablet, connect to the microcontroller, and control the robot over the internet. So how does it work? There’s still some refinements to make, and potentially some mounting brackets to make things more solid, but after various bits of trial and error, it’s a solution that provide a basic telepresence solution, and the basic build cost is under £90 while some interesting additional options may bring the build up to around £100. That’s far, far cheaper than pretty much any alternative around, and it’s been an interesting and fun thing to build, even if some of the problems (such as Skype crashes or trying to find documentation on the Doit.am wifi car app and scripts) have been rather frustrating. The camera on a tablet doesn’t have the ideal field of view for something like this, so it can be hard to see a table that you’re near – it may be worth considering a fish-eye lens attachment depending on whether this proves a problem. As a first serious hardware project, it’s been interesting to see what works. My total build cost has probably been around £40 higher than listed here because of some parts that didn’t work out, but most of those are now either available for other projects or already being used (e.g. a sheet music stand that didn’t prove effective as a mount was quickly claimed by my daughters).

As promised previously, a list of the parts and suggested sources are below. If someone wants step-by-step instructions on the build, modified scripts, Wifi TCP/UDP Controller config file, etc, then by all means ask – it might be interesting to write this up as an Instructable at some point, although perhaps having refined the idea and scripts a little further first. In addition to the components, you’ll need a soldering iron, a drill and ideally a rotary tool.

Component List:

Optional additional components:

  • Lithium-polymer battery pack (7.4v or 11.1v) and battery charger (for extra speed when moving) – £10-15
  • Qi charging pad and receiver (to set up wireless charging) – £7 upwards

Some further progress towards the telepresence robot

My previous effort to build the base literally stalled as the motors proved a little weak for driving the unit (which comes in at around 2kg). In theory the motors should be able to drive this weight, but reality suggested otherwise. As a result, I investigated cheap motors which have a reputation for having a higher torque value and a power option that would provide a higher voltage if necessary.

After another frustrating delay (ordering from China is sometimes surprisingly quick and sometimes ponderously slow), I have some more parts – some GA12-N20 motors with wheels, which are geared to a slower speed (which is good for this project) and apparently provides more torque than the motors on the Doit.am Wifi car. Since I’d ordered a spare microcontroller and motor control board to play with, I figured I’d leave the Wifi car wired up and wire up the spare instead. Below shows the test setup, with helping hands holding the motors, the control board in the background, and a Li-Po battery providing a couple of extra volts and as much current as I need.

Connecting to this via the usual Wi-Fi control app showed the wheels running at a decent speed under load, and grabbing a wheel and the motor I had to use a fair amount of force to stop it turning. Testing with just the USB input didn’t provide nearly the same torque, so at the moment I’m not sure if these motors will work without the added power from the Li-Po – I need to get them mounted on something and add a couple of kilos of weight to test.

My impression is that I now have all the parts I need, but I need to work out whether I’m mounting 2 of these with a dolly wheel, and, if so, whether it’s sufficient with the USB battery pack alone or whether I need the Li-Po as well (which would be a bit more of a pain as I’d want to wire it properly using the larger connector on the battery, so I can put a low-voltage alarm on the 3-pin one for safety and would mean separate charging for the motors when that runs down). Alternatively I got a couple more motors of the original type, so could change to a 4 wheel setup and see how that works, or even whether using the Li-Po with the WiFi car kit as-is gives it enough oomph.

Either way, I’ll have a few bits and pieces to tinker with after finishing assembly, but the total cost of the unit itself shouldn’t change much – at most by a few quid – it should still be possible to build a unit for under £100, so it’s on target still.

Plan B for telepresence robot hardware

After the previous effort at planning a base, where I realized a sheet music stand did NOT make a good support, I had a look at tablet mounts and decided that it would be better to get a lightweight camera tripod and a tablet mount – both together can be had for around £15. This should resolve the issues with how well the tablet is held in the unit, while providing an adjustable height and angle for the display. One order placed, a short wait, and assembly begins again. 

And… now this looks more promising. Some experiments showed that there needed to be a reasonable spacing for the legs to give stability, so the base may need to be wider than I’d originally planned (I’m wondering if this’ll start looking like a mutant Dalek by the time I’ve finished…). With 2 legs extended, however, it’s a reasonable height for seated eye level and a slight upward angle may make it usable for talking to people who are standing, too, so long as I’m not too close. It’s the right height for my younger daughter and my son when they’re standing, anyway. And I can always adjust the height later. The base may require some more work, but I think this’ll do for a stand. The tablet holder also leaves a gap in the middle, so I can plug in a USB cable just fine.

The Wifi car base has arrived, and I’ve got that with me in London to get it fully assembled. I’m reckoning I’ll have to work something out to space the wheels further apart, since the base is significantly narrower than the stable width of the tripod legs. I don’t really want the robot tipping itself backwards when it moves, or for my dog to knock it over too easily.

The instructions for the wifi car were great. At least, if by great you mean only covering a part of the assembly and pointing you to a completely broken version of the application so nothing works. As too often happens with kits like this ordered from the Far East, it’s time to use your Google-fu to find such niceties like a) how to wire up the microcontroller, b) a version of the app that works (and is in English), and c) downloads for updating the controller to work via a remote connection.

Armed with this, assembly was actually pretty easy. The installation instructions showed how to attach the motors and battery pack to the base (the bit I’m most likely to abandon to create a wider base and integrated power supply). After that, I had a packet with a few wires, some random nuts and bolts and spacers, and the microcontroller with motor control board. The first discovery is that the holes on the base don’t really line up with holes in the motor board – you can get it kind of attached using 2 spacers, but it’s kind of balancing above the base and the wires you’re provided with aren’t really long enough to go from that mounting position to the motors – you need the control board positioned somewhere there are no matching holes. My recommendation would probably be double-sided foam tape, but for now I’ve made a double-sided sticky pad using duct tape, which is holding it in place OK. Looking online, you have 2 motor control sets with positive and negative – one pair to each motor, so not too difficult. The default is to share the power supply wires to both the motor control board and the microcontroller – unless you want to use more powerful motors, that’s what you want, so you then need to wire up the ground and Vin to the power supply. Since there’s an on-off switch on the motor board, the kid rocker switch is kind of optional, but it’s a little neater, so I wired that in.

Note that no tools are supplied here. You’ll need a small screwdriver and a soldering iron, which I’m not including in the budget as many will have these. Also some solder, the aforementioned foam tape or other sticking option (hot glue would work for a permanent attachment), and plenty of patience to work with the poor documentation.

Now, it’s time to test out the base. Did it turn on? No. Remember I mentioned the control board has a switch? Well, the ‘instructions’ helpfully didn’t mention that. It was off, so the unit wasn’t working. Once that was on, I’m in business. I see a hotspot to connect to, install the app it suggested, connect to the hotspot, run the app, and… nothing. This is where you need that supply of patience. The APK they recommended? It didn’t work – searching on Google found another version of the app which did, and which gives the option of connecting via a remote connection or local hotspot. The latter is the default. We’ll want the remote connection later, but for testing everything is working, use the local first.

With a better version of the app, it connects, I hit forwards, and it goes forwards and keeps going. I may need to adjust this – you don’t want a telepresence robot to keep going without you, so I may need to tweak the (thankfully open source) microcontroller script to automatically stop after a short time without input. However, it’s moving, both the motors are working, my soldering is better than I’d feared, and we’re in business. Press left and … it turns right. OK, that’s simple, at least – the motors are wired up opposite to what’s needed for the app, so quickly unscrew the motor wire pairs and swap them over, power on again, and it’s moving as you’d expect.

I’m inordinately happy with that – it’s only a crude app-controlled car at the moment, but all the parts work. I’d been wanting to have a USB battery pack which could power both the tablet and the base – I was thinking any twin-socket battery pack would do. I happen to have a nice Duracell one lying around, so I plugged a Micro-USB cable into that, plugged it into the ESP8266 NodeMCU board, and tested the app again. Sure enough, the car moved just as happily powered by this as when powered by the 4xAA batteries. This means that I can definitely have one battery pack power both the tablet and the base. With enough of a battery capacity, this could stay on standby for a week. A twin socket battery pack can be had for around £15, but a large-capacity one would be more.

However, I have another cunning plan. My phone didn’t have wireless charging, so I picked up a Qi pickup that slots into the MicroUSB port and a Qi charger base. The total cost for this is around £11. It can provide 800mA, which should be more than the drain when on standby. My tests suggest that the Qi pickup should be able to charge my battery pack faster than the idle base and tablet drain it, so hopefully I can set up a Qi wireless charging base station whereby I back the robot into the base and so long as it’s close enough it’ll charge. I’m thinking it could be done with the Qi base either on the floor or the wall, but that the wall is likely to be the easier one to get working.

Now, there’s some playing around with the software, and probably a need to assemble a wider base (I could try hot-gluing the motors and dolly wheel to the tripod, but a more solid base would be better to mount the battery pack and microcontroller on and ensure a low centre of gravity). But what’s the cost so far?

Tablet – £39
Tripod – £7
Tablet Mount – £7
Battery Pack – est. £15 (the Duracell one is much more expensive, but I had it anyway)
WiFi Car – £15
Qi Charger – £6
Qi Pickup – £5
Total – £94

So, it’s coming in under budget (particularly since I already had the battery pack). It will clearly be possible to build some kind of telepresence robot for around £100, including wireless charging. For a complete bare-bones option, you could drop the Qi parts and battery pack and have something sufficient for a meeting or other use where someone can turn it on for you for around £70.

My plan is to work out a full parts list with links to appropriate parts, full instructions, once I’ve got things fully assembled and any software and hardware issues resolved. Hopefully others will be encouraged to build their own budget telepresence robots. It’s been an interesting experience so far, planning hardware, doing some soldering, investigating microcontroller software, etc. I’m sure once it’s all working I’ll have a temptation to look at extensions – the most interesting would be looking at some kind of autonomous system which maps out its environment using Simultaneous Location And Mapping (SLAM) or the like, but a good start might be some homing beacon idea along with obstacle avoidance. First things first, though – I need to finish the basic build and deal with any issues like stability, auto-stopping, etc.

Working on a telepresence robot

Since I’m in London for work during the week, and only home with my family at the weekend, I want whatever opportunities I can to be more engaged with my family during the week. I’m also keenly interested in technology – tinkering with code and ideas during my free time.

The natural conclusion for this is to explore the areas of video calling and telepresence. If  I can’t be there in person, then having a more complete means of engaging is of interest. To that end, I’m looking at building a budget telepresence setup – complete with video calling and a mobile base. I want a good quality of video call, I want to be able to be understood reasonably easily, and I want a stable base that I can control remotely, and I want to do it for under £100.

There were a few options considered for the screen and camera elements. I have a Microsoft Life HD webcam which has a pretty reasonable feed, and combined with a Raspberry Pi 3 this would have provided a reasonable amount of processing power, full control over the operating system and a good quality video. There are screens available reasonably cheaply for the Pi, but most of them are small and once you look at 7 inch screens and above the price looks unlikely to fit in the budget. Additionally, while it seemed desirable to have a full open source stack using something like Linphone or another SIP-based video calling software, in practice I didn’t have much luck getting the Raspberry Pi to handle video calling reliably. I’m open to revisiting this approach, but with the budget constraints other alternatives were in order.

The main alternative is a tablet. With the budget an iPad is obviously out. There are some Chinese tablets around for as little as £20, but the battery life was liable to be poor, and the camera and speakers equally so – not promising details for a telepresence base. Doing some research into what tablets had a decent camera for under £50 there were few options – for around £100 there are a few more options, but that’s above the budget. Some (but not all) of the Kindle Fire HD tablets have cameras, and I remembered Amazon making a big thing about the video calling for its support features, so they seemed like a possible candidate. While too expensive new, there are plenty of refurbished models around, so ultimately I decided on the 2012 Amazon Fire HD 7 Inch. These can be picked up for under £40. (N.B. Don’t get the 2013 Fire HD 7 model, as it didn’t have a front camera, so it’s no good for this project.)

Testing one out, they also advertised Skype support. Great – I’ve used Skype lots of times and with auto-answer surely that’s a solution? If only things were that simple. With Skype installed, a new account created, and appropriate auto-answer and volume settings, I tried a few test calls with the tablet on, and everything looked good. However, when I let the tablet go into standby, I found Skype was no longer listening – the fixed Fire HD configuration seems to be to turn off the WiFi to conserve power when in standby. Since I couldn’t leave the screen on all the time, this is a deal-breaker. I tried LinPhone, and it had the same problem. I tried an app that claims to configure this setting, installed from the Amazon App Store, and having paid my dollar, I found it didn’t do a thing. That’s frustrating – I mean the app had ONE function, and it couldn’t do it. So one completely negative review later (such empowerment – at least hopefully it’ll deter others from wasting a few cents as well), it’s time for a more drastic solution – Cyanogenmod.

The Fire HD is a pretty reasonable tablet on its own, but if you have a limitation you don’t like a number of people have worked hard to get a more open version of Android running on the tablets. For a few quid more you can buy a refurbished tablet with Cyanogenmod already installed, and I’d recommend doing so unless you particularly like to tinker, but I already had my tablet, so I had to do the rooting, recovery package installer and Cyanogenmod installation myself. That’s a tutorial in itself, and there’s ones online. For now, I’ll recommend just getting a pre-flashed tablet. With Cyanogenmod installed and Google Apps, I could install a current version of Skype and set the tablet to leave the Wireless connection on in standby. Job done.

The next part of the build was some kind of support for the tablet. I thought it would be clever to get a sheet music stand – figuring it was a suitable weight, narrow at the top for a low centre of gravity, and with a suitable support for the tablet. In short, I was wrong. It served me OK for making a video call to the kids while working on a Plan B, but it’s just not good enough. Firstly, there’s no lip at the bottom and no clip for the tablet, so it’s not secure. Secondly, unless you’ve decided to mount a 14+ inch tablet, the size of the top support is going to be way too big. They’re not really designed to minimise size when you have something smaller, so while the sheet support folded up, it didn’t do so in a way that I could use it, and the tablet was even less secure. I also couldn’t charge the tablet as the charging port is on the bottom and there’s metal from the stand in the way.

I discovered the ESP8266-based Doit.am WiFi car kits online, and figured one of those would make a reasonable base. At the point of trying out the sheet music stand, this was still awaiting delivery, but at £15 for a motorized base which can in theory be controlled by a remote app over the internet, it’s a possible solution to the movement question that will require minimal investment and new development.

So, at the end of the first experiments, my daughter has a new sheet music stand for when playing her recorder, and I had a possible base on the way, a need for a better support plan, and a working tablet. Definitely progress, but a way still to go… tune in again for part 2.

Fingers dancing across the keyboard – professional standards and equipment

One of the frustrations I’ve often had in a workplace is a sub-standard development machine. The net result is lengthy delays while programs switch, code builds, database queries run and so on. While you naturally try to minimise the disruption by thinking about the next steps and doing what you can while things are unresponsive, it can be a serious productivity killer. This invariably strikes as an entirely ridiculous situation given the impact on the business. For the sake of a few hundred extra in hardware per year, a developer whose time is costing many tens of thousands a year is losing a non-trivial proportion of their productivity to just waiting for their machine to catch up. The economics only require 1% of that developers time to be eaten up for it to be better to get a decent machine, and if you’re noticing the delays in various stages of your development efforts you can absolutely guarantee that it’s a lot more than 1% productivity that’s lost.

It gets worse still for the company, though. By being cheap on the equipment that the developer gets to use, this fosters frustrations in the developer. Are the management idiots that can’t see the self-evident value of decent equipment? Do they just not like my enough to me with decent equipment? Or worse, are my prospects being negatively affected by this reduced productivity, as I look like a more expensive resource for a given output, through no fault of my own? The doubts and frustrations are hardly likely to be conducive to productivity in what can sometimes be a stressful job. When you’re thinking about things other than the code, like how to tweak your settings to improve the build speed, or what you can kill to free up a little bit more memory, then that’s mental space taken away from the complex models that you need to keep in mind. And it you have to go make a coffee while things build, then that may be nice for catching up with your colleagues, but doesn’t get any more code written either.

So, the first point of the post is that the equipment developers should have access to should be commensurate with the simple fact that they are undoubtedly a very expensive resource. I’m a professional developer, and I know I cost my clients a lot of money. I aim to provide more than enough to justify that cost, but I’m keen to make myself as valuable as possible to the clients. This involves not just providing input on the best approaches to development, recommendations on viability of options and, of course, the development itself, but suggestions for peripheral improvements to the whole process. £1000 spent on a high-spec machine should be returned several times over in improved productivity over a machine costing half that, when it comes to development machines. Naturally, there’s a point of diminishing returns, as the absolute top-end machines can be astonishingly expensive, but aside from pandering to someone’s ego by making them feel special, it’s going to be rare that you need the absolute best machine going.

This should include other things such as a good keyboard. I have an OK keyboard at work. It has reasonable key spacing and keys haven’t yet got stuck, but it’s a keyboard worth about £10, which seems about standard for most work machines. I’m going to be spending large parts of my working day at the keyboard, whether it’s writing code or documentation or relevant articles about the company or related technology. The difference is less noticeable, just an occasional ache in the wrist from the less than ideal setup, but again, for the sake of a few quid there’s a slight impact on the performance of an expensive resource.

I’d compare it to hiring a consultant surgeon and then providing them cheap or antiquated equipment to perform their procedures with. The risks in a developers case are, thankfully, rarely a matter of life and death, but both will see an impact on their performance given sub-standard equipment, and both are specialists that cost significant amounts to employ – in some cases the developers may be more expensive than the surgeons (I’m not trying to suggest we’re necessarily worth that, it’s a simple product of supply and demand market forces given that most areas of IT are not considered vocational, game development excepted).

There is a corresponding obligation from developers. Not only should they be maintaining their development skills, but they should be ensuring that they are as capable of possible of maximising the time that they spend working out code rather than writing it. This means that since there is a substantial amount of typing involved, some time should almost invariably be spent in learning to touch type. Whether you stick fully with the process after the initial training, the difference is quite marked in the speed that you can write words. In some ways the position isn’t entirely ideal for a developer, as the use of less common symbols may imply a less than comfortable contortion of the hands to stay generally in a touch typists hand position, but the muscle memory and starting point it develops means that you can write code and documentation faster than before. For the sake of a few hours or so adjusting your naturally developed typing process to one that is able to develop further, you’ll reap benefits in terms of the speed at which you can type. By this I mean that a slow developer should be able to type at least 50-60wpm just from their natural typing practice. A practiced developer who has spent some time improving their typing speed should be looking more at a 100+wpm typing speed, comparable with many typists. You’ll develop more of a feel for when you’re typing things incorrectly, reducing the number of times that you mistype a variable name (saving time when you build – remember that we don’t benefit from a spelling checker in most of what we write). The aim is to develop your typing to a point where not only does it reduce the amount of time that you spend writing the code, but it reduces the amount of time you think about the process of typing, and it also allows those moments of flow when your logic is clearest to be expressed as completely as possible, maximising the productivity. If your typing is slowing down your ability to express the code that you’re imagining in your head, then it’s a limiting factor, and those are areas to improve.

More specific to developers are editors and keyboard controls. Most IDEs provide a range of keyboard shortcuts which allow many things to be done more efficiently from the keyboard than from the mouse. The process of selecting menus or buttons is relatively slow compared to a single keyboard combination, not least because you have to take one hand away from your ideal typing position as well as then positioning a cursor in a small part of the screen. If you’re not using a reasonable selection of keyboard shortcuts then you are not maximising your productivity and should take a step back to learn these, as the time they can save as well as the avoidance of distraction to do a task that can’t rely mainly on muscle memory make them something that any professional should look to embrace. Editors like vIM or EMACS can seem like a waste of time to learn with the steep learning curve and lack of full IDE functionality, but the sophistication with which you can interact with the text in a file and their lightweight nature can make them a very efficient way of modifying code. It’s interesting to see Microsoft move more towards supporting lighter tooling with Visual Studio Code alongside the full (and very different) Visual Studio, as this supports a range of plugins that can improve productivity and support a workflow that minimises delays while background tasts freeze up your editor.

There is naturally some downside to the process of getting developers to type much faster. It’s not that time is taken up doing that over learning about new languages or frameworks, because the amount of time involved is small and will be quickly overtaken by the  time saved by typing code and documentation faster. The downside is the liikelihood of taking up more of other people’s time by writing long essays (like this one). Where a short note might previously have sufficed, the ability to type volumes at speed means that a half hour typing up some notes to a meeting can be 6 pages of comprehensive documentation. I’ve had complaints that my emails are excessively long . I’ve rarely spent a long time writing them, I’ve just let something flow. The final step,perhaps, is to get the same developers to spend some time editing what they write for conciseness and form. While this blog is more of a stream of consciousness and not intended to be minutely edited, when I’m writing emails and other communications I try to reduce down to side. I’ve become quite fond of Twitter as a result. This enforces a very small size limit. My approach then ends up being to write what I want to say, delete at least half of it, and then review what I have left and trim it down to size for a Tweet. It’s a good exercise,trying to express a concept in so few letters, although i (as demonstrated here) have yet to consistently apply it to everything I write.

TL-DR: I hope that the general message has been clear. Provide good equipment to developers as even a few percent of time spent waiting for check-ins, builds, debuggers, etc, can cost a company much more over a single year than the additional cost of a well-specced machine. Developers should also optimise their use of equipment – for example learning to touch type and what the most useful keyboard shortcuts are. If both are done the developer is able to maximise their focus on writing code that matches a complex model in their head, improving productivity, staff morale and hopefully also the quality of the code (by reducing delay-related distractions).

Bugbears about proprietary lock-in on hardware

We’ve come to expect printers to require their own ink cartridges, even to the point of having microchips to identify the cartridge. This makes ink cartridges relatively complicated when they really shouldn’t be. This is an area where at least you’re paying extra for the knowledge that it’s the right ink for the printer, so it shouldn’t dry up or clog the printer. The printers may also be relatively subsidised as the manufacturers aim to make their profits from the ink rather than the printer, giving you very competitive prices if you don’t print much.

This is nothing new. Razor blades have been the classic place for vendor lock-in – making no money on the initial purchase but charging a huge premium for branded cartridges compared to the cost of plain razor blades. Again, you buy into the system knowing that there’s a lock-in and you’re paying more for the disposable part and less up front.

What’s much worse is ignoring common standards to provide proprietary interfaces for hardware like hard drives and cameras. By all means implement a driver or app that provides value-added features to improve the experience. But don’t hide away the open standards so people can’t do as much with the hardware.

An example. The Buffalo Ministation Air. You’d buy it if you want 1/2 TB of storage wirelessly. If you don’t care about wireless, then you’d use a wired portable drive instead. So why, then, does it not publicly provide a means to access the drive wirelessly from a Windows device? Why are we locked into a poorly maintained application that limits what files you can see on Android, and which sometimes fails to list even supported files? Why can’t we have Windows shares/Samba, NFS, etc, perhaps DNLA for sharing media? Those would support other applications. Why do the decision-makers assume that all their customers only want to use their devices in the ways they tell them to? I don’t. I want to be able to access the files directly from another app. I want to play files wirelessly from my Windows tablet. I might even want to push the boat out and try to think of it as being properly integrated storage with my other portable devices.

It’s rather disappointing, then, to have the manual effectively tell you ‘this hardware may be sleek and pretty, but it has behavioural problems and doesn’t play well with other kids’. Fortunately, they lie. Of course they’re not going to implement their own alternative to Windows shares. They just keep them hidden away and don’t bother to publish them. That way they can write noddy apps to control the whole process that use that secret wisdom, while we uninformed (presumably unwashed) masses have to satisfy ourselves with the meagre offerings passed on. Or, rise up in glorious revolution, proclaim loudly the secret knowledge, and help others to free themselves from the shackles of app lock-in. 10.10.10.254/Public1. admin/admin. The truth is free.

Another example are some of the IP cameras. There are established protocols which are well supported by drivers, allowing various systems to stream video data from disparate sources to do with what they will. Limited only by creativity, intelligence, technology and (frequently) patience, it’s possible to create a network of open devices monitored all at once, or analysed for movement, faces or Funny Cat Fails. In the name of simplicity (and selling a few premium services at ‘only’ the cost of their brand reputation), many of the IP cameras hide away their open protocols behind applications which require registration or subscription to a monitoring service which can (for a fee) detect movement and host it themselves. If you want to be really stung, you can store a few GB of video, too. And when you want to watch it, you can watch from your Android app, one feed at a time, with motion detection that soon becomes reminiscent of water torture as the incessant reminder that your family are still in and moving around leads you to turn off any such motion detection, unencumbered as it was by the ability to set Appropriate Times Of Day to operate.

I have 4 IP cameras. One of them is a lump of Chinese cheapness, with all the quality instruction and build quality that I’ve come to expect from that. For all that, perhaps because of that, it doesn’t forgo open protocols, it embraces them, and as a result is the most useful of the set. The pair of Philips IP cameras and my BT IP camera are both much more solid feeling. They look nicer. The video is clearer. But I can’t hook them up to a custom system to try to identify who’s at the door because they’re all LOCKED DOWN. Yes, in the name of simplicity I can only access them from an app, and now if I wanted to set up all 4 cameras at once I’d have to switch between 3 pieces of software to view them all. What could make things easier? I think you’ve guessed by now – it would be much easier FOR ME if the manual told me how I could access the underlying open protocols, because then I could (almost trivially) integrate them with the software I want to write. I’m pretty sure the open protocols are there (for some I’ve seen a web page with login running on the camera), but without the required access information, I’m stuck in Toy Land.

These Android apps certainly make simple use cases easier. You don’t need to worry about firewalls and routing, you set up a single account and that’s almost job done. Some even have some nice integration to pick up a QR code from the webcam stream to quickly configure the IP camera to your account. But when you want to do something, anything, out of the ordinary, then these become like training wheels to a professional cyclist – severely restrictive of making full use of what would be possible without those extras. Like training wheels, users should be able to free themselves from the proprietary apps and use the hardware to its full capacity.

So please, any people working for hardware vendors reading this (I can but hope), when you’re looking at a custom app or open protocols, remember. It’s not an either/or proposition. It’s OK to have a ‘my first IP Camera/Hard Drive/Automated Nose Picker’ interface, but unless you’re actively advertising the lock-down, let the underlying protocols be known and accessed so that those of us who want to do more than 1 thing with a device have a reasonable chance to do so. We might even start spouting about the virtues of your open software rather than ranting at length about lock-in.

edX Scalable Machine Learning course review

For me one of the great things about modern society is that high quality education is more accessible than ever before. While the UK now has significant university fees (though still a lot cheaper than many US universities), since Stanford’s AI-Class and ML-Class offerings sites like Coursera and edX have started offering a wide range of courses online in association with a range of major universities (including Stanford, Harvard, MIT, Berkeley and many more). Generally a ‘verified’ certificate is offered for a fee, but you can get a basic certificate following the same video lectures and doing the same weekly assignments, and this doesn’t cost anything.

For someone who is always keen to learn more or learn something new, this is an excellent opportunity. You can learn about subjects outside your career path without worrying whether there’s any financial benefit to doing so, and you can expand your knowledge in your chosen subject in a well-planned and demonstrable way. What better way to demonstrate your wish to excel and to keep your skills updated than by taking university undergraduate and postgraduate level courses in subjects that keep expanding your knowledge?

The latest course I’ve done has been Scalable Machine Learning. It’s a 5-week course that (kind of) follows on from the Introduction to Big Data with Apache Spark. Both were run with Berkely on the edX.org site. Both used Python and Apache Spark via a virtual machine that provided an IPython notebook server for working on the assignments. Introduction was also a 5 week course. The format on edX is that you have a set of video lectures, broken up to chunks of 10 minutes or less. Accompanying this are transcripts, slides and sometimes links to external sites for more information. Each week there are assignments to work on, which are auto-graded, and there are a range of questions covering the information in the lectures and the results from the assignments. 

The course had a Week 0 set of tasks to set up the necessary environment which was identical to the Introduction to Big Data set up. These were easy to follow, and I quickly got an environment together. It assumes little system configuration knowledge and uses a preconfigured virtual machine. You won’t know how to set up a Spark cluster after this, but you’ll have a VM you can explore a smaller volume of data with.This is probably about right for the course, although it would have been nice to at least reference some materials for creating a full-scale cluster – I’ve been getting my own server set up to run Hadoop, HDFS and Spark, and have had to look elsewhere for instructions on creating a physical server installation.

The proper weeks assume no knowledge of Machine Learning, but do assume a good standard of Python programming knowledge. If, like me, your primary programming language has been something else, so your knowledge of things like the Numpy library are more patchy, then additional time is required investigating the relevant libraries. I should say I’ve written a CMS in Python, albeit a while ago, so my Python skills were rusty more than non-existent, but I’d had little exposure to the Numpy library and the matrix manipulation functions required here, and while occasionally I had to dig around for the correct way to do something. Most of the time it didn’t really get in the way, and between the Introduction course and this I’ve gained a lot of confidence with Python’s take on lambda functions and for comprehensions along with the NumPy library knowledge and PySpark-specific details.

Week 1 explores a range of core details, such as complexity issues (Big O notation), a bit of an introduction to NumPy and matrix manipulation in Python, an overview of Apache Spark and a run-through of some core Machine Learning concepts like the difference between supervised and unsupervised learning. This was the foundation week, bringing everyone up to speed on the array of concepts that were going to be used in the rest of the course. Much of this was already familiar from other courses I’ve taken on Machine Learning, but for those who haven’t taken other courses it seemed like a clear and reasonably scoped explanation of the key concepts. I mainly learned some more about the NumPy library here.

Week 2 introduces Apache Spark, giving more detail on why it’s useful, how it aims to outperform the likes of Hadoop and what makes it different to processing the data at small scale in something like R. It introduces the core data structure – RDDs – and the concepts of transformations (which aren’t run immediately) and actions (which trigger the preceding steps), culminating in implementing some logic to process text data. This was a duplicate of one of the weeks from Introduction to Big Data, with the same assignment, so those doing the 2 courses together as part of a logical sequence of courses had an easy week here. After this you know how to build up a reasonably complex piece of logic through a pipeline of transformations, how to cache intermediate results, and how Spark splits, collects and processes the data.

Week 3 moved on to the Machine Learning side, looking primarily at Linear Regression. Here I was impressed by the quality of the course, as it not only explained how to use the tools, but started with the principles of Linear Regression, closed form solutions, solving some simpler problems without the Spark libraries. It moved to how the solution can be performed in a way that can be performed at scale to give an understanding of how Spark can solve the problem in a distributed manner, then solved some more problems using plain Spark code. Having demonstrated the principles, then it moved to using the libraries, ensuring an insight into What Spark is doing and How it is doing it, not just How to use its libraries. This also introduced the concept of grid search and hyperparameter tuning. While there are other parameter tuning strategies which I would have liked to see discussed further, it seemed a reasonable scope for this course. Similarly, while it didn’t really explain in detail how quadratic features may help in certain circumstances, it did explore how to generate these and demonstrated that they did have an impact on the prediction accurate for the data being explored.

Week 4 covered Logistic Regression, specifically looking at things like Click Through Rate prediction. Here the course moved onto something quite relevant for many interested in data analysis. It covered how to handle things like categorical (non-numeric) input variables, training a classifier to indicate whether something fits a criteria, and how to improve your model. Again, it started with the mathematical underpinnings of Logistic Regression, and walked through the various data processing steps involved in more real-world data processing, like trying hashing, One Hot Encoding, hyperparameter tuning and interpretation of the results using an ROC plot. Prediction accuracy was covered, not just by getting a value, but explaining how you might need to consider False Positives over False Negatives or vice versa and how to measure whether your model is any good by comparing to simple models (like predicting the most likely outcome). This week again went into good depth with the theory while also covering key aspects that are sometimes glossed over but which are necessary for getting a good working model.

Week 5 wrapped up the course with an explanation of some basic concepts about Neuroscience, why its problem falls into the Big Data category, and why dimensionality reduction is so necessary for interpreting data. Having outlined the domain knowledge required (with some nice video illustration of the relevant images that will be processed), it then moves into dimensionality reduction. This walks you through the mathematical functions to build up Principal Component Analysis from the basic matrix functions, and applies it to a dataset to process the images in a number of ways. In this it takes what could be some matrix calculations whose function is relatively unclear and helps you to understand how the various different matrix calculations can perform an efficient aggregation and how PCA allows massively complex data to be reduced to something that can train models or be visualised. In this is probably demonstrated PCA as more useful and understandable than any other explanation of it I’ve seen.

Throughout the lab work was at times challenging, but generally when I got stuck it was only because I was tired and hadn’t read something properly – coming back with fresher eyes was generally the solution. There’s a lot to take in on the course, and the knowledge from it would be useful in Data Mining and Machine Learning. I was very much impressed with the standard of the course, the scope, the pace, and how it demonstrated how the concepts can be applied to real data. My main complaint would be that it could have covered more on the ETL kind of aspects – data extraction, transformation, loading (and cleaning) – it covers a little early on but could be expanded on. On the whole, though, it’s a very high standard of course, and certainly the match of many or most paid university modules.

Data Protection Act and Anonymity

The current course I’m running through (Scalable Machine Learning) and other recent online courses (including Introduction to Big Data with Apache Spark) are amongst the many high-quality courses available on edX, Coursera, Udacity and so on. Prestigious universities offering free online courses so anyone can learn about advanced topics, not just in IT but in other subjects as well.

There is one disappointment with the current module, though. As it looks at click-through rates and notes that the source data has been anonymised, it’s obvious that while the technical side is being covered in good depth (covering the mathematical foundations, basic implementations, process for scaling to a distributed setup and the functions to use), there is far less coverage of the ethical and legal aspects. This is perhaps to be expected from such a short course (this course should be around 20-25 hours of study), However, a little more discussion would be good, if only enough to raise more awareness of the issues.

In legal terms, it may be easiest to set up moderately large-scale data processing in the cloud initially. It’s generally cheaper to run the servers in the US than in Europe. However, if you transferred personally identifiable data from Europe to the US then you risk falling foul of data protection legislation. What’s currently largely untested is what potential liability you have if supposedly depersonalised data proves to be sufficiently specific to identify individuals, but without touching on some of the basic ethical considerations about data privacy students may not be encouraged to find out their obligations or consider the rights and wrongs of this.

There is a point where personalisation could potentially become creepy or intrusive. If you searched for a pregnancy test, then you might not want to receive other pregnancy-related results – particularly on a shared computer which might make your condition apparent to others who you might not wish to tell yet. Similarly, targeting that may imply or show someone as having a stigmatised condition such as HIV, or lead to questions about sexual, religious or other views could in certain circumstances cause significant harm.

In the pursuit of better targetting, better classification, better modeling, we need to make sure that what we do is not just commercially justifiable but ethically and legally justifiable as well. Those of us who are registered professionals (of BCS or another recognised professional group) have an obligation in our professional code of conduct to operate in an ethical and legal manner. This means that we should raise awareness of these issues if they arise, and, if necessary, to refuse to perform work which is in breach of those standards.

I don’t take this view lightly, and have sought to raise legal concerns which have arisen in previous contracts. It’s not always easy. As a contractor without the protections of a permanent employee, that may mean risking your income to disagree with decision-makers. However, like an expectation to perform some pro bono work to provide service back to your community, our professional obligations aren’t just about what is best for us personally, but about what is honest and right. Many IT managers haven’t studied the legal and ethical concerns previously, so there is scope to raise these issues, generally.

As we make more and more use of the data we have available, we have to consider whether correct permission has been given for the use of the data in this way, whether it is being held in the correct locations, whether the correct access restrictions are in place, and what the correct response would be to a Freedom of Information request. These are just a few of the considerations. They apply not just when gathering the data, but in how it’s used, and if you look carefully, various advertising and personalisation systems offer options to only have more generic results and turn off the better personalisation. This should be an option available in your own site if you’re looking at something like recommendation systems based on past purchases or item views.

It’s a complex area, and the legal details go beyond my limited understanding of the legislation (I’m not a lawyer), but perhaps the ethical questions can be as much of a guide as the legal ones? Is this personal? Would I want my data used in this way? Am I acting for or against the best interests of the person or people whose data I’m analysing? Those are perhaps easier questions to answer without studying endless legislation, but no less important if you want to do things right.

Giving back to society – Pro bono work for IT professionals

One of the things that I recognise is that my line of work is very well paid compared to many – I may have to put in a lot of my own time keeping my skills current and developing knowledge in new areas, but in return I get satisfying work with good financial rewards. In line with expectations on Lawyers and the like, it is not unreasonable to think that at least the better-paid IT professionals should also look to do an element of pro-bono work to contribute back to the community.

While I am willing to help someone set up their computer, replace hardware, reinstall an operating system, etc, that’s not really what I do day to day, it’s just something I’ve learned along the way. However, where a friend helps me with some decorating or transport or the like, I’ll pay them but will not expect payment for helping fix their computer, it’s part of how I can give a little back. Much as there are geek T-shirts announcing they will not fix your computer, I think many of us can recognise that we have specialist knowledge and can help others who would struggle, and that by virtue of having a lucrative career we have a responsibility to give something back. If you want to donate to charity, then that’s naturally fine with me, but if you’re not able or willing to do that, then providing some of your spare time isn’t necessarily that big a request.

That’s the most commonly identified demand for IT experience. Most developers will not expect to write a complete website for a charity without being paid – if nothing else, the likelihood of getting ongoing time demands to support the site may prove problematic. Simpler requirements are more feasible, though – I’ve set up a club site or 2, but would generally stick to just a few pages, or help set up a basic CMS or e-commerce solution so they can then work on the site themselves. The advantage of this kind of work is that you can potentially take the opportunity to get experience with technologies or areas in which you’re less experienced, and can generally get permission to point a potential client at such work, so this kind of work has some potential rewards anyway.

However, perhaps the most important aspect where IT professionals can help their community is by helping promote best practices, awareness and mentoring. The first 2 involve trying to ensure that you’re doing your bit to improve the quality of code, offering advice on best practices, discussing and shaping standards, and promoting awareness of professional bodies like BCS with their codes of conduct and commitment to professionalism in IT. The mentoring side is rather separate, but given that there are ongoing calls for more people coming into the industry and a shortage of skills holding back companies, mentoring can help those with the requisite knowledge get those key first steps and positive experiences within the industry in order to build their own positive career.

My brother likes it when he has the opportunity to hire someone with little commercial experience and provide solid on the job mentoring to instill a range of best practices on top of the basic development skills already learned, to ensure each new team member quickly becomes a productive team member capable of working efficiently in a team. For myself, as a contractor I have far less opportunity for mentoring inexperienced team members currently, but am always happy to share knowledge with co-workers and provide mentoring if appropriate. Instead, I will happily invest as many hours as required into discussing options and plans for people directed my way to gain knowledge about programming and to get their first steps in the field. The first thing I can offer suggestions on is CV structure – as a contractor my CV has probably been reviewed more often than most people’s, and has been continually revised. On an IT note, I’ve put massive amounts of time into continuing professional development, so I (like most good programmers) know many resources for learning, including good quality free online courses, and have a feel for the technologies that are in demand.

Any experienced programmer would be in a position to offer some guidance on how we specialise in particular areas – and not to worry about learning everything in the subject domain, as it’s just not possible. We can point novices at Stack Overflow and other invaluable resources. We can provide some tips about how to approach programming (e.g. save early, test often, because it’s so much easier finding that bug in a few lines of code than a few hundred… and reassure them that even after years in the field logic mistakes are a common occurrence, the skill comes in spotting them earlier and more reliably and not letting them out into the wild). We can provide some ideas for projects to work on, and look over some code to suggest ways in which it could be improved. And when they get stuck in what they’re trying to learn we can provide some suggestions on how to resolve their current problem. It may even be refreshing to find that you have someone who is willing to adjust the way they write things and take on board the advice, when most of us have experience working with at least some developers who have lax development practices and no willingness to improve.

Young adults have much higher levels of unemployment than older working-age adults, and getting that initial step into a field can be daunting. A surprising proportion of graduates who are working 6 months after graduation are not working in their field of study. By providing mentoring and support of those you know who are keen to get into the field, we can help people in a challenging moment in their lives, and help them build a satisfying and rewarding career. A few hours of our time to help a friend (or, more often, a friend of a friend that you’ve never met before, in my experience) isn’t really that much, and I think most of us are pretty happy to talk about technology and IT at the drop of a hat anyway. It’s unlikely to get any reward other than personal satisfaction at having done something worthwhile, but that’s kind of the point. Pro Bono work is about giving something back, not about doing something in the hope of getting further rewards for yourself. And mentoring (rather than general IT support) allows our specialist knowledge to be passed on and shared within our community, to hopefully improve the availability of such knowledge and capability to more people indirectly as well. If they ask what they can do in return, the best answer seems to me to ‘pay it forward’ – rather than trying to pay back the help you’ve received, agree to make similar efforts to help someone else in need, so someone else deserving help can get the support they need.