OpenCL: .NET, C# and Resolver One integration -- the very beginnings

Today I wrote the code required to call part of the OpenCL API from Resolver One; just one function so far, and all it does is get some information about your hardware setup, but it was great to get it working. There are already .NET bindings for OpenCL, but I felt that it was worthwhile reinventing the wheel -- largely as a way of making sure I understood every spoke, but also because I wanted the simplest possible API, with no extra code to make it more .NETty. It should also work as an example of how you can integrate a C library into a .NET/IronPython application like Resolver One.

I'll be documenting the whole thing when it's a bit more finished, but if you want to try out the work in progress, and are willing to build the interop code, here's how:

• Make sure you have OpenCL installed -- here's the NVIDA OpenCL download page, and here's the OpenCL page for ATI. I've only tested this with NVIDIA so far, so I'm keen to hear of any incompatibilities.
• Clone the dot-net-opencl project from Resolver Systems' GitHub account.
• Load up the DotNetOpenCL.sln project file in the root of the project using Visual C# 2008 (here's the free "Express" version if you don't have it already).
• Build the project
• To try it out from IronPython, run ipy test_clGetPlatformIDs.py
• To try it in Resolver One, load test_clGetPlatformIDs.rsl

That should be it! If you want to look at the code, the only important bit is in DotNetOpenCL.cs -- and it's simply an external method definition... the tricky bit was in working out which OpenCL function to write an external definition for, and what that definition should look like.

I've put a slightly tidied version of the notes I kept as I implemented this below, for posterity's sake; if you're interested in finding out how the implementation went, read on...

[ Read more ]

OpenCL: first investigations with an NVIDIA card

I'm taking a look at OpenCL at the moment, with the vague intention of hooking it up to Resolver One. In case you've not heard about it, OpenCL is a language that allows you to do non-graphical computing on your graphics card (GPU). Because GPUs have more raw computational power than even modern CPUs, in the form of a large number of relatively slow stream processors, this can speed up certain kinds of calculations -- in particular, those that are amenable to massive parallelisation.

Until recently, the two main graphics card manufacturers had their own languages for this kind of general-purpose GPU computing; NVIDIA had CUDA, and ATI/AMD had their Stream technology. OpenCL was created as a way of having one language that would work on all graphics cards, so although the tools for developing using it are not currently as good as those for CUDA (which has been around for a long time and has great support), as a longer-term investment OpenCL looks to me like the best one to be looking at.

It took a little bit of work to get something up and running on my machine here at work, so it's probably worth documenting to help others who are trying the same.

[ Read more ]

Playing with NLTK

I spent some time this afternoon playing with NLTK, the Python Natural Language Toolkit; the book is highly recommended. I couldn't resist sharing the results of the first time I tried its generate function, which takes a particular text and tries to automatically generate more text in the same style.

First, from the Book of Genesis:

In the selfsame day entered Noah , and asses , flocks , and Maachah . And Joseph said unto him , Abrah and he asses , and told all these things are against me . And Jacob told Rachel that he hearkened not unto you . And Sarah said , I had seen the face of the air ; for he hath broken my covenant between God and every thing that creepeth upon the man : And Eber lived after he begat Salah four hundred and thirty years , and took of every sort shalt thou be come thither .

Next, Jane Austen:

[ Sense and Sensibility by Jane Austen 1811 ] CHAPTER 1 The family of his relations , I know Marianne ' s indisposition , beyond giving time for the world to seeing YOU .-- Very far from being weary of her feeling how more than commonly joyful . He was four years . Every thing was disregarded . I had called in Berkeley Street , January . " Mrs . Jennings , sagaciously smiling , " For my part , not being Willoughby , where much of his fate with Miss Morton , with two pointers playing round him ,

And finally, from a corpus of IRC conversations:

now im lost i agree i saw the whole thing JOIN PART google 's start page .. lol yeah , he said this morning he is hoping they win tonight im gay ....... im happy as hell ...... jus aint #### lol U42 , how are you NICK : U23 what a combo hehehe JOIN . ACTION pictures the blues brothers behind that chicken wire screen . (((((((((( U34 ))))))))))))) Hi U7 ......... how are ya ll gotta watch my manners or she wo n't you play another somebody done somebody wrong song ? JOIN . ACTION wonders if U16

Scarily accurate :-)

London Financial Python Users' Group

I clearly need to post more stuff here so that it doesn't just turn into a blog announcing the LFPUG's meetings :-)

However, in the meantime, here are the details of the next one: it'll be on 11 March 2010, and is hosted this time by Man Investments Ltd at Sugar Quay, Lower Thames Street, London EC3R 6DU. As before, all are welcome, but for security reasons you need to register in advance; just drop an email to Didrik Pinte. (Update: old mailto link removed.)

Guest of honour this time around is Travis Oliphant, the creator of SciPy and the architect of NumPy. He'll be talking about NumPy memory maps and structured data-types, and Didrik will also give a talk about integrating C/C++ libraries using Cython. More suggestions for talks (or even better, offers to give talks!) are very welcome -- once again, just email Didrik, or post something in the LinkedIn group.

Next London Financial Python Users Group meeting

The next meeting of the London Financial Python Users Group will be on Feb 3, 2010 at 7pm, and is being kindly hosted by KBC Financial Products at their offices: 111 Old Broad Street, EC2N 1FP (just opposite Tower 42).

All are welcome, but for security reasons you need to register in advance; just drop an email to Didrik Pinte. (Update: old mailto link removed)

The topics planned for this meeting are:

• Improving NumPy performance with the Intel MKL - Didrik Pinte, Enthought
• Python to Excel bridges:
  • "PyXLL, a user friendly Python-Excel bridge" - Tony Roberts
  • Discussion on connecting Python and Excel (xlrd/xlwt, pyinex, win32com, pyxll, ...)
• Speeding up Python code using Cython - Didrik Pinte, Enthought

New laptop!

Back at the end of October, I asked whether when I bought a new laptop I should get one with a Core 2 Duo T9600 or a i7 720QM. They both cost the same, and there seemed to be pros and cons to each.

After weighing things up, and particularly with the advice of Greg Bray in the comments there, I'd pretty much decided on the i7, so on 2 November I was ready to buy. I went to the Dell shop... and discovered that they'd added GBP400 or so onto the price for both models in the spec that I wanted. Welcome to the pre-Christmas price rise.

So I waited until early January, and finally the price for the i7 returned to where it had been (though the T9700 was still expensive). And last Thursday, the new machine arrived. i7 720QM quad-core at 1.6GHz (up to 2.8GHz when only one core is active), 4Gb RAM, ATI Mobility Radeon HD 4670 with 1Gb RAM (which can do 432 GigaFLOPS[!]), Windows 7 Pro 64 bit, 256Gb SSD, and a 16" RGBLED screen (if I understand that correctly, the backlight is an array of LEDs in groups of 3, and it adjusts the backlight colour in a screen segment to match the colour in that part of the screen). All for the same price as the cheapest 15" MacBook Pro.

It's lovely. The build quality is excellent -- the only comparable machines I've tried have been the MacBook Pros and the Air, which are nice but slower and considerably more expensive. (To be fair to Apple, the MacBook Pros are better-made than this Dell. But it's a close thing, and I don't think it's worth the extra. YMMV.) The screen is the best I've seen since I saw high-end CRTs on SGI Workstations back in the 90s.

Size and weight-wise, it's comparable to a MacBook 17" -- smaller, but in the same category. So I wouldn't recommend you get one as a netbook -- but as a desktop replacement, it really looks like it's going to be perfect.

The only thing that sucks is the face recognition login widget, which has so far managed to recognise me once. Out of 30 tries. But hey, it can be uninstalled.

Joining TheyWorkForYou to Twitter

I'm putting together some spreadsheets that we're going to use to publicise Resolver One over the coming UK electoral campaign, and one set of data I needed was a list of Members of Parliaments' Twitter accounts indexed by an ID that I could use with TheyWorkForYou.com. I was delighted to discover Tweetminster, a site analysing MPs' tweets, and in particular their Twitter list of all UK MPs' accounts, but there was no link to TWFY.

So, given that no-one else seems to have done it, here's my own list:

• A Resolver One file listing Twitter accounts for TheyWorkForYou person_ids
• For those of you unlucky enough to not have a copy of Resolver One, here's a CSV file with exactly the same data.

Hope someone finds it useful. It's up-to-date as of this posting, and I'll endeavour to keep it up-to-date, at least for as long as we need it at work :-)

IT headhunters considered harmful

I got an interesting call from a headhunter today; he knew that we were likely to start hiring software developers at Resolver Systems soon (keep an eye on our jobs page or drop me a line if you're interested) because he had helped someone who'd chosen to leave us to find their new job.

As I said, it was interesting. I admire his honesty if not his morals; while most such people will merely hint about things, this chap came straight out with it: "we're actively trying to poach people who work for you, and we'll stop doing it if you stop trying to recruit people on the open market and use us instead".

[ Read more ]

New gadget: Nokia N900

I got an early Christmas present today! Robert was able to find a Nokia N900 at a very reasonable price, and Lola decided to get it for me as combined Christmas and birthday presents...

The is a fascinating device. Nokia bill it as a "mobile computer"; it has about the CPU power of a 10-15-year-old PC (say, a Pentium II), but also has a modern graphics processor, and it runs Maemo, Nokia's rebranded version of Debian Linux. Earlier this month, Robert showed me his one running his own port of Frotz, and demonstrated that it had a full version of Linux; I saw the video of it running WebGL later on the same evening, and fell in love :-)

One thing I've noticed, though, is that although you can get a full Debian install running, it takes a little work and there's no good step-by-step documentation. I'll post about that next.

A website for LFPUG

Didrik Pinte has put together a web page on the Python.org Wiki for the London Financial Python Users Group. Only a little content so far, but it will grow... if you're doing financial work in Python in London, do come along to the next meeting -- it will be 7pm next Monday (14 December) at MWB Regent Street, Liberty House 222 Regent Street, London W1B 5TR. You may have to put up with me talking for a while about a spreadsheet you already know everything about, but there will be interesting bits too ;-)