Thursday, December 24, 2009

Getting the most out of Python range()

I am doing a small data analysis project and figured it was a good time to learn a bit of Python. One of the requirements was that my data analysis range go from a start to a finish value in certain increments. The range() built in function was perfect for this, except that it stops prior to the last value. For example, for range(10, 25, 5), the values 10, 15, 20 would be produced. I needed 10, 15, 20, 25.

The obvious fix for this is to add the increment value to the stop value: range(start, finish+increment, increment). Unfortunately, this falls flat when the difference between your stop and start values is not a zero modulus of the increment value. For example, range(10, 23, 5) would produce the values 10, 15, 20, 25 based on the obvious fix, and that goes past the finish value.

The proper fix is the following (broken up into two lines for readability):

mod = increment - ((finish - start) % increment)
range(start, finish+mod, increment)

The astute will notice that all this "magic" really does is bumps up the finish value so that the difference between the finish and start is a zero modulus of the increment. For situations where the difference between the finish and start value is already a zero modulus of the increment, this solution effectively implements the "obvious fix" above.

The same thing could be done by simply adding the increment value to the finish value only when the difference between the finish and start value is a zero modulus, but that requires an "if" statement to detect that situation. My solution replaces multiple lines of code, or an awkward looking conditional (does python even do conditionals???), with a nice clean mathematical statement that works for all situations.

Tuesday, November 24, 2009

Theories and Laws...

I am amazed at the number of people who do not know what the word "theory" means... Maybe this will clear it up.

Hypothesis - educated guess that may or may not be backed up by facts. When most people say "in theory" they really mean "I hypothesize that...".

Theory - A way of explaining why something works the way it does, backed up observational/experimental results. The Chilton's guide for my car is the theory manual behind the operation of my car. The theory of gravity attempts to explain why gravity works the way it does. A book on computer science is the theory guide explaining how your computer works. And so on...

Law - A description of a phenomenon, but does not explain the "why". A train schedule is an example of a "law". It explains when the train will arrive but has no power to explain why the train got there on time. The "law" of gravity explains why your head hurts when someone drops a brick on it, but only the "theory" of gravity explains why the brick dropped downwards, instead of just floating there in space.

Last but not least, in case it is not obvious, theories are never promoted to "law" status. They are totally different things.

Theories explain "why", laws explain "what"...

Wednesday, October 28, 2009

A sed keeper...

For those of you who are shell scripters, you may find this one useful:

/bin/sed ':a;N;$!ba;s/\n\n\n*/\n\n/g'

This removes multiple blank spaces in a file and replaces them with only one blank space. There are other ways to do this, but this is one of them...

Source: http://linux.dsplabs.com.au/rmnl-remove-new-line-characters-tr-awk-perl-sed-c-cpp-bash-python-xargs-ghc-ghci-haskell-sam-ssam-p65/

Sunday, September 20, 2009

Mac Mini Upgrade and Adventures in Boot Camp

Now that Snow Leopard is out and I have some time off between classes, I am able to get Bailey's computer fully upgraded. I thought it was going to be a simple software upgrade, but as it turns out, his Mac Mini was only running on 512MB of RAM and a 80 GB hard drive. Since he is starting to use this machine for more than just games, I guess it is time to max out the RAM and get him a decently sized hard drive.

A quick trip to Best Buy and $200 later, we have 2 GB of PC2-5300 RAM and a nice Seagate 7200 RPM 320 GB hard drive. I considered going larger on the hard drive, but my time was too short to wait for something to be delivered and the 320 GB was the largest drive they had that was 7200 RPM. Also, now that we are going to have three people on the Time Capsule, I figured 320 GB was a good limiter to make sure he would not prematurely force me to upgrade the Time Capsule to 2TB.

I used the iFixIt instructions for opening up the Mac Mini. It was surprisingly easy. One thing they did not mention was to label each screw you remove. I used a sticky note and labeled where I removed each screw from. Most of the screws were identical, but I noticed one or two were specialized sizes, so I was glad I kept that organized. Another part I had to learn the hard way was the RAM insertion. I did not insert one of the sticks hard enough and when I booted up the machine only one stick registered. After re-seating the RAM, I was able to get both 1GB sticks to properly register. I might also add that the Mac Mini is a really impressive sample of engineering. Everything fits together amazingly well. Total hardware time would have been about an hour if I had seated the RAM correctly the first time. Given my mistake, it was around 2.5 hours (I tested each stick individually).

Now that the hardware was working properly, it was time to install Mac OSX 10.6 (Snow Leopard). The install was effectively effortless. There were only two hitches. The first was the basic install. No drives showed up to select as a target to install. I was able to go to one of the installer pull-down menus (I think it was Utilities) and select disk tools to partition and format the disk. Once that was done the installer showed the target disk and the install went through flawlessly.

The second hitch came after the installation was complete. I noticed that the "Macintosh HD" icon was missing from the desktop. After some help from TheGoogle, I figured out it was simply a default configuration option that is delivered with Snow Leopard. To add it back, just click on the desktop anywhere so you display the Finder menu. Select "Preferences" from the Finder menu and make sure "Hard Disks" is checked in the "General" menu. I also found it helpful to go to the "Sidebar" tab and and check the box next to "Hard Disks" and turn off the one for "iDisk" (I choose not to use Mobile Me, so iDisk is not useful to me).

Once all of that was complete I went to Software Update and installed all of the updates (at this point there were only three). Voila, fully working Snow Leopard system with 4x the RAM and 4X the Hard Drive space. Total "from scratch" OS X Snow Leopard install time with updates, 2-ish hours.

Last but not least, it was time to get the WindowsXP partition up and running. Boot Camp has gotten incredibly easy to use since I originally played with the beta. I found the Boot Camp icon in Application/Utilities. It started up and allowed me to partition and format the disk, then it had me insert my WindowsXP Pro SP2 disk and rebooted into the XP installer. One of the biggest problems I encountered was that my keyboard would not work. After some searching, I found out that it was the Mighty Mouse that was confusing the XP installer. I removed the mouse and moved the keyboard to the USB port just below the audio port (the last one), and after rebooting, the XP installer worked just fine (I am not sure that moving the USB keyboard port helped, but I figured I would mention it). When it came time to select the partition to install to, it was easy to find the BOOTCAMP partition. I opted to change the format from FAT to NTFS (I have since verified that NTFS is still read-only from OS X, hopefully Apple fixes this some day). Once the graphical installer interface came up, I was able to plug the Mighty Mouse back in and it worked for the remainder of the install.

After the base XPSP2 install was complete, I inserted the Snow Leopard disk to install all of the Apple specific drivers. Once that was done, it was time to start applying XP updates. I made the mistake of attempting to install the SP3 update first. I kept on getting error messages about not being able to find files like "osloader.ntd". I figured this was because I did not apply the mountain of pre-SP3 updates first, which I then proceeded to do. It took so long I went to bed and resumed the next day. Even after that, SP3 still did not apply cleanly. After some searching, I was able to find this link that perfectly explained what was wrong and provided a solution that worked. After that, SP3 applied cleanly and then I was able to apply the final group of updates. Total time spent getting Windows XP installed and fully updated was probably 12 hours, nearly 3x what was required for a RAM, Hard Drive and OS X Snow Leopard install (4x if I had gotten the RAM installed correctly the first time).

I also spent some time getting FireFox 3.5 installed in each OS as it is one of the required browsers for Bailey's online Math class. The last thing I did was to get The Snow Leopard partition connected to the Time Capsule. This was the easiest part. I clicked the Time Capsule icon in the dock, slid the switch from "Off" to "On" and typed in the Time Capsule password. After that, I just left the machine alone while it did the initial backup of the OS (8.11GB). I should also mention that the first Time Capsule backup has to complete uninterrupted, so I turned off the power-saving settings in case the Mini went to sleep before it was done (I left myself a note to remember to turn it back on when it was done).

Thursday, August 20, 2009

Chiropractic Therapy

(Note: this is the infamous article on chiropractic that got Simon Singh sued. It is being reposted all over the web by multiple blogs and online magazines.)

Some practitioners claim it is a cure-all, but the research suggests chiropractic therapy has mixed results - and can even be lethal, says Simon Singh.

You might be surprised to know that the founder of chiropractic therapy, Daniel David Palmer, wrote that "99% of all diseases are caused by displaced vertebrae". In the 1860s, Palmer began to develop his theory that the spine was involved in almost every illness because the spinal cord connects the brain to the rest of the body. Therefore any misalignment could cause a problem in distant parts of the body.

In fact, Palmer's first chiropractic intervention supposedly cured a man who had been profoundly deaf for 17 years. His second treatment was equally strange, because he claimed that he treated a patient with heart trouble by correcting a displaced vertebra.

You might think that modern chiropractors restrict themselves to treating back problems, but in fact some still possess quite wacky ideas. The fundamentalists argue that they can cure anything, including helping treat children with colic, sleeping and feeding problems, frequent ear infections, asthma and prolonged crying - even though there is not a jot of evidence.

I can confidently label these assertions as utter nonsense because I have co-authored a book about alternative medicine with the world's first professor of complementary medicine, Edzard Ernst. He learned chiropractic techniques himself and used them as a doctor. This is when he began to see the need for some critical evaluation. Among other projects, he examined the evidence from 70 trials exploring the benefits of chiropractic therapy in conditions unrelated to the back. He found no evidence to suggest that chiropractors could treat any such conditions.

But what about chiropractic in the context of treating back problems? Manipulating the spine can cure some problems, but results are mixed. To be fair, conventional approaches, such as physiotherapy, also struggle to treat back problems with any consistency. Nevertheless, conventional therapy is still preferable because of the serious dangers associated with chiropractic.

In 2001, a systematic review of five studies revealed that roughly half of all chiropractic patients experience temporary adverse effects, such as pain, numbness, stiffness, dizziness and headaches. These are relatively minor effects, but the frequency is very high, and this has to be weighed against the limited benefit offered by chiropractors.

More worryingly, the hallmark technique of the chiropractor, known as high-velocity, low-amplitude thrust, carries much more significant risks. This involves pushing joints beyond their natural range of motion by applying a short, sharp force. Although this is a safe procedure for most patients, others can suffer dislocations and fractures.

Worse still, manipulation of the neck can damage the vertebral arteries, which supply blood to the brain. So-called vertebral dissection can ultimately cut off the blood supply, which in turn can lead to a stroke and even death. Because there is usually a delay between the vertebral dissection and the blockage of blood to the brain, the link between chiropractic and strokes went unnoticed for many years. Recently, however, it has been possible to identify cases where spinal manipulation has certainly been the cause of vertebral dissection.

Laurie Mathiason was a 20-year-old Canadian waitress who visited a chiropractor 21 times between 1997 and 1998 to relieve her low-back pain. On her penultimate visit she complained of stiffness in her neck. That evening she began dropping plates at the restaurant, so she returned to the chiropractor. As the chiropractor manipulated her neck, Mathiason began to cry, her eyes started to roll, she foamed at the mouth and her body began to convulse. She was rushed to hospital, slipped into a coma and died three days later. At the inquest, the coroner declared: "Laurie died of a ruptured vertebral artery, which occurred in association with a chiropractic manipulation of the neck."

This case is not unique. In Canada alone there have been several other women who have died after receiving chiropractic therapy, and Edzard Ernst has identified about 700 cases of serious complications among the medical literature. This should be a major concern for health officials, particularly as under-reporting will mean that the actual number of cases is much higher.

If spinal manipulation were a drug with such serious adverse effects and so little demonstrable benefit, then it would almost certainly have been taken off the market.


Simon Singh is a science writer in London and the co-author, with Edzard Ernst, of Trick or Treatment? Alternative Medicine on Trial. This is an edited version of an article published in The Guardian for which Singh is being personally sued for libel by the British Chiropractic Association.

Sunday, August 02, 2009

Palm Pre and a few warts...

I indulged in a Palm Pre a few days ago. I had been waiting to get an Android phone, but I figure it will be at least two years until we see anything Android from Sprint, so locking myself into a two year agreement with the Pre seems like a reasonable compromise.

So far, I really like it, it is beautiful and has a lot of potential. I am still getting used to how things work and I cannot wait to get an SSH client on it and figure out how to tether it.

One thing I did not notice in any of the reviews is that there are a few fairly glaring bugs and annoyances that I will try to detail here:
  • The biggest annoyance is migrating calendar and contacts with the migration tool (I used the Mac OSX version). I started from my old Treo 680 and sync'd it one last time to iCal and AddressBook. After running the sync tool to move things over to the Pre, none of the street addresses get migrated and fax numbers occasionally get mixed up and duplicated. I had to go through my entire address book and manually enter the street addresses. Although very time consuming, I feel that it was a good exercise because it gave me a chance to clean up my address book. Something that I have not done in years apparently... A lesser annoyance is that all day calendar events with no specific time (like an anniversary) show up on the Pre as timed events lasting the entire day. I had to go in and manually change the setting to "All Day".
  • When updating the calendar, it takes an unusual amount of time for the change to actually appear on the calendar. It feels like there is a refresh bug of some sort that keeps the old calendar information on the screen *JUST* until the point at which you are unsure if the change took, and then *poof* the screen updates. Rather strange...
  • The text messaging app does not show character count. This is rather a pain when twitting because there is a 140 character limit... less than the text messaging limit...
  • It would appear as if certain apps cannot be deleted unless I root my pre. I will be doing that eventually, but with the limit of three launcher pages, it would be nice to be able to tidy things up a bit.
  • Call me pathological, but I would like an easy way to migrate over my phone calling history. The Treo was great about keeping a permanent record that would follow you from phone to phone. This does not appear to be the case with the Pre.

I think there are a few other things, but this is all I have for now. For comparison, if I were to list everything I *LIKE* about the Pre, I would be typing all night long. Other than not knowing how many characters in a text message, the rest of the issues can be worked around or are simply not that important...

Friday, July 10, 2009

NASA 747 pilot tells of carrying the Space Shuttle


Reposted with permission. Fascinating story!

Triple Nickel - NASA Pilot

Well, it's been 48 hours since I landed the 747 with the shuttle Atlantis on top and I am still buzzing from the experience. I have to say that my whole mind, body and soul went into the professional mode just before engine start in Mississippi, and stayed there, where it all needed to be, until well after the flight...in fact, I am not sure if it is all back to normal as I type this email.

The experience was surreal.

Seeing that "thing" on top of an already overly huge aircraft boggles my mind. The whole mission from takeoff to engine shutdown was unlike anything I had ever done. It was like a dream...someone else's dream.

We took off from Columbus AFB on their 12,000 foot runway, of which I used 11,999 1/2 feet to get the wheels off the ground. We were at 3,500 feet left to go of the runway, throttles full power, nose wheels still hugging the ground, copilot calling out decision speeds, the weight of Atlantis now screaming through my fingers clinched tightly on the controls, tires heating up to their near maximum temperature from the speed and the weight, and not yet at rotation speed, the speed at which I would be pulling on the controls to get the nose to rise. I just could not wait, and I mean I COULD NOT WAIT, and started pulling early.

If I had waited until rotation speed, we would not have rotated enough to get airborne by the end of the runway. So I pulled on the controls early and started our rotation to the takeoff attitude. The wheels finally lifted off as we passed over the stripe marking the end of the runway and my next hurdle (physically) was a line of trees 1,000 feet off the departure end of Runway 16. All I knew was we were flying and so I directed the gear to be retracted and the flaps to be moved from Flaps 20 to Flaps 10 as I pulled even harder on the controls.

I must say, those trees were beginning to look a lot like those brushes in the drive through car washes so I pulled even harder yet! I think I saw a bird just fold its wings and fall out of a tree as if to say "Oh just take me". Okay, we cleared the trees

duh, but it was way too close for my laundry. As we started to actually climb, at only 100 feet per minute, I smelled something that reminded me of touring the Heineken Brewery in Europe...I said "is that a skunk I smell?" and the veterans of shuttle carrying looked at me and smiled and said "Tires"!

I said "TIRES??? OURS???" They smiled and shook their heads as if to call their Captain an amateur...okay, at that point I was. The tires were so hot you could smell them in the cockpit. My mind could not get over, from this point on, that this was something I had never experienced.

Where's your mom when you REALLY need her?

The flight down to Florida was an eternity. We cruised at 250 knots indicated, giving us about 315 knots of ground speed at 15,000'. The miles didn't click by like I am use to them clicking by in a fighter at MACH .94. We were burning fuel at a rate of 40,000 pounds per hour or 130 pounds per mile, or one gallon every length of the fuselage! The vibration in the cockpit was mild, compared to down below and to the rear of the fuselage where it reminded me of that football game I had as a child where you turned it on and the players vibrated around the board.

I felt like if I had plastic clips on my boots I could have vibrated to any spot in the fuselage I wanted to go without moving my legs...and the noise was deafening. The 747 flies with its nose 5 degrees up in the air to stay level, and when you bank, it feels like the shuttle is trying to say "hey, let's roll completely over on our back," not a good thing I kept telling myself. SO I limited my bank angle to 15 degrees and even though a 180 degree course change took a full zip code to complete, it was the safe way to turn this monster.

Airliners and even a flight of two F-16s deviated from their flight plans to catch a glimpse of us along the way. We dodged what was in reality very few clouds and storms, despite what everyone thought, and arrived in Florida with 51,000 pounds of fuel too much to land with. We can't land heavier than 600,000 pounds total weight and so we had to do something with that fuel. I had an idea...let's fly low and slow and show this beast off to all the taxpayers in Florida lucky enough to be outside on that Tuesday afternoon.

So at Ormond Beach we let down to 1,000 feet above the ground/water and flew just east of the beach out over the water. Then, once we reached the NASA airspace of the Kennedy Space Center, we cut over to the Banana/Indian Rivers and flew down the middle of them to show the people of Titusville, Port St.Johns and Melbourne just what a 747 with a shuttle on it looked like. We stayed at 1,000 feet and since we were dragging our flaps at "Flaps 5", our speed was down to around 190 to 210 knots. We could see traffic stopping in the middle of roads to take a look. We heard later that a Little League Baseball game stop to look and everyone cheered as we became their 7th inning stretch. Oh say can you see...

After reaching Vero Beach, we turned north to follow the coast line back up to the Shuttle Landing Facility (SLF). There was not one person laying on the beach...they were all standing and waving! "What a sight" I thought...and figured they were thinking the same thing. All this time I was bugging the engineers, all three of them, to re-compute our fuel and tell me when it was time to land. They kept saying "Not yet Triple, keep showing this thing off" which was not a bad thing to be doing.

However, all this time the thought that the landing, the muscling of this 600,000 pound beast, was getting closer and closer to my reality. I was pumped up! We got back to the SLF and were still 10,000 pounds too heavy to land so I said I was going to do a low approach over the SLF going the opposite direction of landing traffic that day. So at 300 feet, we flew down the runway, rocking our wings like a whale rolling on its side to say "hello" to the people looking on! One turn out of traffic and back to the runway to land...still 3,000 pounds over gross weight limit.

But the engineers agreed that if the landing were smooth, there would be no problem. "Oh thanks guys, a little extra pressure is just what I needed!" So we landed at 603,000 pounds and very smoothly if I have to say so myself. The landing was so totally controlled and on speed, that it was fun.

There were a few surprises that I dealt with, like the 747 falls like a rock with the orbiter on it if you pull the throttles off at the "normal" point in a landing; and secondly, if you thought you could hold the nose off the ground after the mains touch down, think again...IT IS COMING DOWN!!!

So I "flew it down" to the ground and saved what I have seen in videos of a nose slap after landing. Bob's video supports this! Then I turned on my phone after coming to a full stop only to find 50 bazillion emails and phone messages from all of you who were so super to be watching and cheering us on! What a treat, I can't thank y'all enough. For those who watched, you wondered why we sat there so long.

Well, the shuttle had very hazardous chemicals on board and we had to be "sniffed" to determine if any had leaked or were leaking. They checked for Monomethylhydrazine (N2H4 for Charlie Hudson) and nitrogen tetroxide (N2O4). Even though we were "clean", it took way too long for them to tow us in to the mate-demate area. Sorry for those who stuck it out and even waited until we exited the jet.

I am sure I will wake up in the middle of the night here soon, screaming and standing straight up dripping wet with sweat from the realization of what had happened. It was a thrill of a lifetime. Again I want to thank everyone for your interest and support. It felt good to bring Atlantis home in one piece after she had worked so hard getting to the Hubble Space Telescope and back.