Category Archives: Uncategorized

Less is more for Machine Learning inputs

Leave a reply

I came across some good advice on maintenance of Machine Learning algorithms. The short version, Less is More when you are deciding what data to feed into your algorithm.

It also reminded me of some systems issues Jaron Lanier pointed out in a recent interview.

  • Over time will your model be primarily processing its own predictions?The Netflix movie recommendation engine has narrowed its available inputs because most users only choose between the top predictions.
  • Will your prediction model cannibalize its source of data?Machine learning powered language translation is automating the work of many human translators, whose previous work provides the input needed by the machine learning algorithms.

Never, ever do anything important in Windows

Leave a reply

Just learned a great Microsoft Windows lesson.

There is a database program I like that I use in a Windows VM.  I was in the middle of  executing some commands when: BAM, windows update and restart.

What brilliant design Microsoft, of course the update should just suddenly take control mid-keystroke.

Those commands I was in the middle of could have been something critical. For instance when switching over from one system to another, the last step is often to quickly run a couple of commands to execute the changeover so that there is just a second or two of downtime.

“Ok, tell the request router to stop sending millions of users to the old system.”
“Now, tell it to point at the new… <WINDOWS UPDATE>”

So, never, ever do anything important in Windows.  And change your settings to not automatically apply updates.

Age Adjusted Median Income

Leave a reply

The other day my wife mentioned that median income in the US had gone down over the past few years, which led to a random thought while I was driving home.  Perhaps when median income is plotted over time it should be weighted based on the relative earning power of the current age of the population.  When a large portion of the population is at the age of their optimum earning power (around 50) you would expect median income to be higher than when a large portion is very young or very old.

http://www.census.gov/hhes/www/income/data/historical/household/

At least it’ll make a good pair programming scenario next time I’m interviewing a candidate.

Today’s Challenge: Build a better system during the training meeting

2 Replies

Had a mandatory training meeting today on our time tracking software. Since we already had to figure out the 33 step process a few weeks ago so we could get paid…  today we had an impromptu hackathon.

This software is so bad that surely I can build something better during the training meeting for said software.  -Me

Now, how to beat my coworker Mikkel who is wicked good? We both have to vaguely pay attention to the webinar so that’s an equal handicap. Aha, don’t acknowledge that I’m serious until about 15 minutes in. Especially since my computer is creakily working toward four years of service so a few tasks that should take 20 seconds instead drag on for several minutes. It’s about five minutes into the hour when I start, in theory that leaves 55 minutes to build something cool.

Ok, grails create-app timetrack.
Create a user domain class to sub in for a real authentication plugin.

Now the biggest problem with the real time-tracking software is that you have to enter hours for each day.  This is dumb when the only useful purpose it serves is to track vacation time and try to avoid everyone being on vacation at once during a big release.

Don’t want an hours worked for the day object, let’s do the opposite and create-domain-class TimeOff.  Next create some controllers, set scaffold=true and voila we’ve the world’s simplest app for entering time off.

One of those commands takes much longer than it should and after firing up Intellij as well I’m at the 15 minute mark.  Mikkel realizes I’m serious and starts cranking out a rails app. 

No problem, time for a secret weapon: Dojo.

I know Dojo has some great calendar widgets and a calendar sounds like a good interface for something concerned with days and time. Start looking through docs; not that calendar; this looks right; nope wasn’t that either.  Ok, finally got the right code, still getting a weird error though…

And the webinar is over, done after 40 minutes.

What?  I’ve only been coding for 35 mins and Mikkel for 20.  Well, time to show our results.  Mikkel’s lets you enter hours worked one day after another.  With about 15 steps to fill out a two week pay period it’s a more than 100% improvement over the same part of the real app.  And mine?  Giant javascript error, typical dojo.  Damn.

Couldn’t let it end there though.  Put another 15 minutes in to wire up json output in Grails and get the javascript error fixed and…  Bam, a decent prototype for the interface.

timetrack

For comparison, here’s the real app.  Now I just need to turn my prototype into a real app…

Submit Time Sheet Express Page

How to strangle productivity

Leave a reply

Step 1 Outlook Web App.  Step 2 Time Tracking software.

I’m not quite sure why companies adopt time tracking software.  A friend mentioned that he thought there was some accounting treatment that encouraged it but some web searching has only lead me to BS statements like these.

“Time is money, so once it is clear where employees are spending their time, an employer can decide how to better prepare and deal with projects.”

“With a distributed company it ensures that everyone is working.”

Yes, entering eights into boxes ensures that I’m working.  Apparently if the time tracking interface is as cumbersome as possible someone can be extra sure that I’m hard at work.

Now, HR people tend to be extremely nice and well-meaning, perhaps this makes them especially susceptible to the sales pitch of time tracking companies. I understand if a company has part-time employees, or is billing hours to clients; then tracking those hours can make sense. For salaried employees who are otherwise being treated as adults, it’s a gigantic waste of time. Particularly if the software requires a training meeting.

Which leads to TODAY’S CHALLENGE.

Decision Making

Leave a reply

How a team makes decisions determines how effective that team is.

All teams have a decision making process whether explicit or implicit and all members of the team have a hand in it. Even in a purely a command and control structure, such as a fast food restaurant where process is centrally decided, every member of the team at the restaurant is reinterpreting the decision and evolving it into what actually happens.

Modern organizations do knowledge work where decisions are the primary output: whether to have a sale or run an ad campaign; which feature to build next. Therefore teams need to be highly concerned with the effectiveness of those decisions. This may take a large amount of time and certainly takes effort and focus. Since every member of the team will evolve what they are working on toward what that member wants, a big portion of the decision making process may be mostly invisible.  This hidden misalignment is a big part of why outputs differ from what was decided in the ‘the meeting’ on what those outputs will be.

That’s not to say that everyone at a company needs to be involved in every decision. However, visibility about what decisions are going to be made and when, can allow people to self select to be involved in the decisions that most affect their work. Make it clear that anyone can attend any meeting and that it is everyone’s responsibility to exclude themselves from meetings as well.

Meetings are frequently inefficiently run.  Which is a big problem.  So solve it.  Make meetings short and focused.  If you have set up the meeting, you may need to exclude yourself from arguing a side of the decision to be made in order to keep the meeting on task.  If that is the case, you can appoint a surrogate to help convey your point of view, or appoint a surrogate to run the meeting.  Is there more to discuss?  Have another short meeting tomorrow where whoever still needs to be heard should attend and everyone will have had time to consider and refine their ideas.  Communication needs to happen and it is best if it is explicit and visible to anyone who wants to know.

Another symptoms of bad meetings is when communication primarily happens in informal groups.  Part of this is natural, a huge amount of communication and productive thinking happens over beers or while chatting about something else.  That is good and worth encouraging.  What is bad is if the only forum for people to voice their concerns or for strategies to be realigned is when enough people happen to get together for lunch or drinks.

Want your team to make better decisions? Here are a few questions to ask yourself and your team members.

What decisions do you need to be involved in that you are not now? Which do you not need to be involved in?

Who should be making the tradeoff between X and Y (often the customer if possible), how do we convey the reasoning behind that tradeoff to everyone who is working on it?

Help me out, what other good questions should a team be asking in order to make effective decisions?

Upgrading to Dojo 1.7.2

1 Reply

Upgrading a project to dojo 1.7.2 from 1.6.0.  While I could simply point to the new version (it seems quite backward compatible), I think it’s time to switch to the new AMD compatible require / declare syntax.

Step 1.  Pointed at the CDN and set async to true:

<script type="text/<span class=">// <![CDATA[
// <![<span class="hiddenSpellError" pre="">CDATA</span>[
// ]]></script>
javascript" src="//ajax.googleapis.com/ajax/libs/dojo/1.7.2/dojo/dojo.js"
// ]]></script>
 data-dojo-config="async: true, parseOnLoad:true">
</script>

Immediately the async: true caused dojo is undefined errors from dojo.byId and dojo.require statements.

// A slide out tab
dojo.require("dojo.fx");
dojo.addOnLoad(function() {
    dojo.animateProperty({
        node:"slideOutBox",
        properties: {
            height: 0
        }
    }).play();
});

The fix was pretty easy

Step 2. Change dojo.require to require and move dojo.addOnLoad into

require([“dojo”,”dojo/fx”, “dojo/domReady!”], function(fx){
});

 

Upgrading custom dojo build to 1.7.2

First build overwrote dojo.js, oops, which gave this unhelpful error,

js: uncaught JavaScript runtime exception: ReferenceError: "require" is not defined.

Note: I should have finished this post a year ago when I was doing the upgrade.  There were a few more issues I ran into but it’s a bit fuzzy now.

Gigabyte P55-USB3 motherboard with 2000 MHz RAM

Leave a reply

Just plugged 8GB of G.Skill 2133 mhz ram into my desktop.  The memory clock speed stayed at 1333 mhz when I did so which led to some fiddling with bios settings.

I have a Gigabyte P55-USB3 motherboard with an i5 2.8 Ghz processor and plugged in 2 x 4GB modules of the F3-17000CL11D Ripjaws ram from G.Skill (in a nice red color to boot).  Motherboard manual is here, http://www.gigabyte.com/products/product-page.aspx?pid=3440#manual

Once in the bios config in the ‘MB Intelligent Tweaker’, the ‘Advanced Memory Settings’ seemed to have what I needed, ‘Extreme Memory Profiles’ for different speeds.   Profile 2 put the speed at 2130 mhz and all the advanced timing settings matched the memory specs (11-11-11-30 2N).  Unfortunately, it wouldn’t boot.  Switching between the different  profiles (disabled, profile 1, profile 2) and ‘Performance Enhance’ modes (standard, turbo, extreme) didn’t help much, the only way  I could boot was at 1600 Mhz in the disabled profile with extreme performance enhance.

Fortunately I hit on a nice overclocking guide, http://www.overclockers.com/3-step-guide-overclock-core-i3-i5-i7/

Leaving the memory settings at the default values, I  went into the ‘Advanced Frequency Settings’ section and changed the Base Clock BCLK to 200Mz with  the CPU Clock Ratio at 16x.  This gave me an overclocked CPU speed of 3.2 Ghz (seemed reasonable and a nice bonus) and put the Memory Frequency at 2000 MHZ (good enough).

After a few hours of serious programming work, the system has been completely stable and seems a bit faster.  Hope that helps.

Running your Government

Leave a reply

In the absence of InnoVoter / StickyVote, you may be wondering, “how can I best manage my Government?”  I’m happy to report that this area is still a hotbed of startup activity.  Although several contemporary projects have shut down, some have morphed and new ones have started.

One of the new, Votizen.com, raised $1.5 million just over a year ago and has quite the who’s who of advisers and investors.  Interestingly they have chosen to focus on voter registration records; one of the big early decisions I made was just to prototype that piece and to leave full implementation for later.  Before too long we should see whether that will form a strong foundation for Votizen to build on or a crushing weight of software and data maintenance.

Next on the list is PopVox.com, co-founded by the guy behind govtrack.us, which is still a great source of Federal legislative data.  To communicate with Congress this is the best tool I’ve seen, they have great widgets and do a good job of grouping similar bills together.  The one other I recommend is OpenCongress.org, a non-profit with a strong community of users commenting on bills as well as any easy place to find general information on Congress.