Quick links

What caught my attention recently:

• The latest Google and Microsoft earnings show damage from a tech disruption, a shift to mobile that is impacting both badly but for different reasons. Google needs to crack mobile ads. Microsoft needs to get share in mobile computing. ([1] [2] [3] [4])
  
  
• Now "there are almost as many mobile phone subscriptions in the world as people" ([1] [2] [3])
  
  
• Google is getting aggressive, releasing a $99 tablet and a $250 laptop ([1] [2] [3] [4])
  
  
• Amazon prices their tablet at cost ([1] [2])
  
  
• And decent tablets in the $50 range are already widely available in China ([1])
  
  
• But Microsoft prices its new tablet above the cost of an iPad. ([1] [2])
  
  
• Meg "Whitman liberally mixed metaphors to describe her awakening to just how screwed HP was" ([1])
  
  
• "Prepare for Windows 8 induced user rage" ([1])
  
  
• "The argument that C.E.O.'s will leave if they aren't compensated well, perhaps even lavishly, is bogus" ([1])
  
  
• "FTC puts a bounty on the heads of robo-telemarketers" ([1])
  
  
• On Amazon EC2, testing performance of the instances and rejecting ones with weak performance can make a huge difference ([1] [2])
  
  
• Good article in The Atlantic about the considerable lengths Google is willing to go to increase the quality of Google Maps ([1])
  
  
• "We read Apple's secret Genius Training Manual from cover to cover. It's a penetrating look inside Apple: psychological mastery, banned words, roleplaying—you've never seen anything like it." ([1])
  
  
• Big data is "a process that uses data to refine our thinking. But it doesn't work without some thinking first." ([1])
  
  
• Surprisingly detailed talks on Netflix and LinkedIn's recommender systems ([1] [2] [3])
  
  
• Great talk on A/B testing, especially how to do A/B testing at large companies ([1] [2])
  
  
• Amazing speaker list at a workshop on big data for personalized education, slides from many of the talks are available ([1] [2])
  
  
• Sometimes research just confirms what we already know (or should know), in this case, that simpler websites with familiar themes in the design do better ([1] [2])
  
  
• "Savvy Internet users know that all the great stuff they get from the Internet us for 'free' -- the searches, the social networks, the games, even the news -- isn't really free. It's an exchange, where companies are able to take user data, sell it to advertisers, and make money." ([1] [2])
  
  
• In the US, "80% of teens ... have a game console" ([1] [2])
  
  
• Meanwhile, in Estonia, "a new education program that will have 100 percent of publicly educated students learning to write code" ([1])
  
  
• Xkcd on dinosaurs ([1])
  
  
• Good TED talk on publication bias, which is caused by not publishing negative results ([1])
  
  
• Got willpower depletion? One study claims, if you believe willpower depletion exists, it does, otherwise it doesn't. ([1])
  
  
• "Ever heard of the marshmallow test? The outcome may have more to do with conditioning from a child's environment" ([1])
  
  
• "Is playtesting essential to making a good game? Yes ... [But] playtesting is like an engraved invitation that reads: 'You are cordially invited to tell me why I suck. Bring a friend - Refreshments served.' The whole point of playtesting is to make clear to you that some of the decisions you made ... are completely wrong." ([1])

Read More

Code Monster and teaching programming to kids

I recently launched Code Monster from Crunchzilla. It helps parents teach a little programming to their kids.

A lot of parents want their kids to learn a little about programming. But, if you are a parent, there seem to be only two choices out there, either have your kids slog through all the syntax and pain of tutorials and textbooks made for adults, or have them learn a visual programming language made for kids that can't be used for anything else.

Code Monster teaches Javascript, which is a useful and valuable programming language to know. When learning using Code Monster, the code is live, so changes kids make have impact immediately. They learn a bit about how to program, starting with early concepts like parameters, variables, and loops, moving through functions, eventually introducing some of the wonders of fractals, animation, and physics. Code Monster encourages experimentation. It makes programming fun.

Code Monster is an unusual blend of a tutorial and a game. It is not a tutorial or a lesson plan, but it does walk kids through many experiments with a real, useful programming language. It is not a game, but many of the children who have playtested it have found it fun, addictive, and exciting.

If you're a geek like me, there are some techie aspects of Code Monster you may find interesting. For example, Code Monster uses live code so kids see the immediate impact from code changes, no hitting a run or compile button. Code Monster provides useful help messages if the player stops working on the code but has an error. There are several nice but subtle features -- like preventing most accidental infinite loops -- that are harder to do than you might think (if you think you know how to do that in Javascript, try it, I bet your solution doesn't work). It only needs an internet connection when you first go to Code Monster (allowing working on lessons offline) and keeps your progress without saving anything remotely (privacy is important). The lessons eventually introduce quite sophisticated topics -- like fractals, L-grammars, animation, and physics -- that are very fun for kids but not normally taught to beginning programmers. But all of that tech stuff only matters because it makes Code Monster do the right thing; the important thing is that Code Monster fun and enjoyable to use.

Code Monster came out of my interest in online education, especially math and computer science education. I am convinced that, when this generation of children grows up, algorithmic thinking, large scale data analysis, and programming will be a major force multiplier for people working in many fields. People who have these tools will have the power to find breakthroughs in medicine, biology, economics, and many other areas; these tools will let them do things no others have done. I hope Code Monster can be a small piece of many more girls and boys becoming interested in computational thinking.

Please try Code Monster. It's free and it's fun. If you have kids (especially ages 9-14) , please have them try it. If you know people who have kids (or adults who are young at heart and might want to dabble in programming), please tell them about it (and share on Facebook, Google+, and Twitter too). I'd love to get the word out about it, and it's all for a good cause, it's teaching kids to program. Finally, if you have any suggestions or find it useful for your kids, please post a comment here or e-mail me at monster@crunchzilla.com, I'd enjoy knowing how you like it and how I can make it better.

Read More

Will tablets replace PCs?

I just bet Professor Daniel Lemire $100 that they won't.

At least, any time soon. The specific terms of the bet are, "In some quarter of 2015, the unit sales of tablets will be at least twice the unit sales of traditional PCs, in the USA." Loser donates $100 USD to the charity of the winner's choice.

How did I get to this point? About a year and a half ago, I wrote a blog post for CACM, "Who needs a tablet?"

The purposely inflammatory title overstates the main point, which is that rather than replace PCs, people are mostly buying tablets in addition to their PC ([1] [2]).

Even so, predictions in the article have already proven wrong. Tablet sales did not "stall around the same level where netbook sales stalled". Netbook sales peaked and stalled around 40M units/year worldwide ([1] [2]). Tablet sales passed 60M units/year worldwide in 2011 and are projected to be twice that this year.

So, tablets show no sign of stalling where netbooks did, but they are still being bought in addition to, not in replacement of, PCs. While many are taking some of the time they would have spent on their PC and spending it on their mobile or tablet instead, they still own and spend time on a laptop or PC.

This bet doesn't quite say what I want to say. What I want to say is that PCs aren't going away any time soon. They definitely are not going away by the end of 2015. Eventually, yes, but the change is not going to happen in less than three years.

What the bet actually says is more about how fast people in the US will buy new tablets in 2015 compared to replacing PCs. Projections I've seen put PC unit sales in the US around 16M units/quarter and mostly flat through 2015, tablet unit sales currently at 7M/quarter in the US and growing rapidly (projections vary from 10-16M/quarter by 2016). Seems unlikely that the projections would be that far off, so I took the bet.

But the more interesting questions are:

1. What will it take to get people to stop using PCs?
2. Will the tablet market continue to be dominated by expensive devices (like the $600 iPad) or convert almost entirely to low priced tablets (currently $200 with the Nexus 7 and Kindle Fire, but probably soon around $100)?
3. Will anything coming in the next five years, including tablets, get people to stop buying and using PCs entirely? Or will people continue to buy and use multiple computing devices?

I've said what I think (breakthroughs in input/output, almost all $100 tablets, no). What do you think?

Read More

Quick links

Some of what has caught my attention lately:

• Pump and dump, both at the Facebook and Groupon IPOs. ([1] [2])
  
  
• "The thrilling demise of Groupon's crummy business model" ([1] [2])
  
  
• Dave McClure says, "Returns for venture capital 'absolutely suck' ... even worse ... most VCs are 'insufferable, arrogant, fucking assholes'." ([1])
  
  
• And good advice here, also from Dave McClure: "Don't do a startup, you idiot!" ([1])
  
  
• Remember all the startups in desktop search a few years ago? They all disappeared when Microsoft fixed desktop search in Windows. Likewise, cloud storage is increasingly becoming part of the operating system (in MacOS, Windows, and Ubuntu), and that likely will kill off startups like Dropbox. ([1] [2])
  
  
• This is the end of the customizable home page hype, also a popular startup idea a few years ago ([1])
  
  
• "Once valued at more than $160 million, [Digg] is selling for the deeply discounted price of about $500,000" ([1])
  
  
• I wonder why we don't see engineers leave en masse for another company. Lack of organization? Fear of being sued?([1])
  
  
• After saying "Windows 8 is terrible for desktops", a reviewer goes on to predict, "Windows 7, with its 630 million licenses sold, will remain an incredibly popular OS for the next 10 years -- just like Windows XP." ([1] [2] [3])
  
  
• WinXP amazingly still has 26% market share but, in a bizarre twist on top of that, Microsoft decided not to support IE9 on WinXP; WinXP users have to use Chrome or Firefox if they want a modern browser. ([1] [2])
  
  
• Brutal (and long) Vanity Fair article on Microsoft. To summarize, stack ranking and Ballmer's repeated errors killed confidence, morale, and the company's performance in the last decade. This quote captures the dysfunction: "People responsible for features will openly sabotage other people’s efforts. One of the most valuable things I learned was to give the appearance of being courteous while withholding just enough information from colleagues to ensure they didn't get ahead of me on the rankings." ([1] [2] [3] [4] [5])
  
  
• Microsoft has a decent phone out now, but it's priced so high, no one sees the point of getting it. You can't have a product consumers see as inferior to an Apple product but charge Apple-level prices, people will just get the Apple product. ([1] [2])
  
  
• Others had the same idea as the iPhone, just no one but Apple was willing to piss off the carriers and partners and launch it ([1])
  
  
• A change that may have widespread impact, current smartphones are getting powerful enough that people are waiting longer before replacing them; they're happy with what they already have. A similar thing happened a while ago with PCs, with dramatic impact on that industry, could be just starting for smartphones. ([1])
  
  
• Of course you can sacrifice customer service in the short-term to boost short-term profitability. Customers take a while to learn that the service is not what it once was; you're essentially drawing down from past investment in your brand. After a few years, your brand becomes soiled, retention rates fall, customer acquisition costs rise, and profitability plummets. This has happened many times in the past, and is happening again right now. ([1] [2] [3])
  
  
• A hybrid recommender, using both content and behavior data, wins A/B tests on Forbes.com articles. Why does that sound familiar (cough, Findory, cough)? ([1] [2])
  
  
• Cute idea, default local search results not to where you are, but where you are likely going, based on your current trajectory ([1])
  
  
• Nice example of how better hardware in your database can be faster and cheaper than expanding your caching layer ([1])
  
  
• What we introverts have to go through to act like extroverts ([1])
  
  
• If you have ever worked with software engineers and thought, "Why are they so grumpy?", this article provides insight, understanding, and solutions. ([1])
  
  
• Long article from Steve Yegge, but with some thought-provoking points about liberal (risk embracing) and conservative (risk avoiding) programmers. ([1])
  
  
• A start on personalized education, recommendations for courses ([1])
  
  
• An interesting difference between Coursera and Udacity is that Udacity is sticking mostly to computer science. I think Udacity is right to do so, but also curious how well Coursera manages to do in fields outside of CS. ([1])
  
  
• Love DragonBox, a game that is primarily a fun puzzle game, but also teaches algebra. The math is subtle; the puzzles involve matching and moving things on two sides of the screen that, as it turns out, represent two sides of the equation and all your moves are the same as moving things between two sides of an equation. Great for kids, really fun and addictive to play, love this, more like this please. ([1] [2])
  
  
• Nice example of A/B testing in the physical world ([1])
  
  
• Google App Engine launched at the top of the stack (write code and don't know where or how it is running) and Amazon EC2 at the bottom (just providing virtual machines). It's been interesting to watch both of them move toward each other, Amazon launching more and more features on top of EC2 (like CloudFront and Elastic MapReduce) and Google launching lower level services (like this new move to allow you to run your own virtual machine in Google's cloud). ([1] [2] [3] [4] [5] [6] [7])
  
  
• Why read research papers? "These papers often foreshadow where the rest of the world is going." ([1])
  
  
• I like this search quality metric, WTF! @ k. Colorfully useful. ([1])
  
  
• Google Research and their hybrid research model blends research and engineering (to maximize impact and avoid the problematic tech transfer from research) and keeps projects short (but still do long-term research by iterating). ([1])
  
  
• Cow Clicker is a very amusing (and bizarrely successful) deconstructive satire of Zynga games, reduced to just clicking, waiting, and buying your way out of waiting, hilarious. Also worth seeing is Nekogames' Parameters, which breaks down Diablo-like games to their core elements. ([1] [2] [3])
  
  
• NNet guru Geoffrey Hinton says, "The brain is confronted by a buzzing, blooming confusion. It needs to fit many different models and use wisdom of the crowds." He then goes on to show the surprising benefits of massive NNets that drop out hidden units randomly. ([1] [2])
  
  
• "We oversimplify because, simply, there is no other way of getting by in the world" ([1])
  
  
• "It's not that our memory is a glitchy wetware version of computer flash memory; it’s that the computer metaphor just doesn't apply ... We store only bits and pieces of what happened—a smattering of impressions we weave together into feels like a seamless narrative. When we retrieve a memory, we also rewrite it, so that the time next we go to remember it, we don't retrieve the original memory but the last one we recollected." ([1])
  
  
• Amazing technology, a camera fast enough to catch light moving, can see around corners using clever algorithms, well worth watching this short talk ([1])
  
  
• Another amazing technology, very clever algorithms allowing an autonomous plate to fly at high speed in a constrained space. Go robots! Well worth watching this too, also short. ([1])
  
  
• Yet another impressive video, worth watching. Simple idea that breaks an assumption, solves a long standing problem with robot grippers, very effective, clever. ([1])

Read More

The computer scientist CEO

Marissa Mayer as CEO of Yahoo may be a test of a new style of executive leadership, the optimizing CEO.

She is not the first computer scientist to lead a major company, but she is the first computer scientist (MSCS or higher) hired in as CEO to a Fortune 500 company. Many computer scientists view everything as an optimization problem. People, work, politics, life, everything is a search (often of a dynamic space) to find a maximum near the global maximum.

Marissa Mayer is an important test of a new style of CEO. She is not a Neutron Jack or Carly Fiorina, the strong military general style of bold decisions, loyalty-first, follow me, right or wrong. She is not going to be a charismatic cheerleading, press-focused CEO, the type that views their job solely as managing the message and marketing and selling the company and themselves. She is not going to be the mad visionary of Steve Jobs, yelling at everyone while single-handedly designing breakthrough products. She is a computer scientist and appears to be leading like one. I suspect she views the company, people at the company, the products, even her own role, all as an optimization process, a search to find the most productive and most useful outcomes.

The most common degree of CEOs hired into Fortune 500 companies is an MBA. Marissa appears to be the first computer scientist. This may be a test of a new style of leadership. Will Marissa Mayer be the start of companies hiring optimizing CEOs?

Read More

Puzzling outcomes in A/B testing

A fun upcoming KDD 2012 paper out of Microsoft, "Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained" (PDF), has a lot of great insights into A/B testing and real issues you hit with A/B testing. It's a light and easy read, definitely worthwhile.

Selected excerpts:

We present ... puzzling outcomes of controlled experiments that we analyzed deeply to understand and explain ... [requiring] months to properly analyze and get to the often surprising root cause ... It [was] not uncommon to see experiments that impact annual revenue by millions of dollars ... Reversing a single incorrect decision based on the results of an experiment can fund a whole team of analysts.

When Bing had a bug in an experiment, which resulted in very poor results being shown to users, two key organizational metrics improved significantly: distinct queries per user went up over 10%, and revenue per user went up over 30%! .... Degrading algorithmic results shown on a search engine result page gives users an obviously worse search experience but causes users to click more on ads, whose relative relevance increases, which increases short-term revenue ... [This shows] it's critical to understand that long-term goals do not always align with short-term metrics.

A piece of code was added, such that when a user clicked on a search result, additional JavaScript was executed ... This slowed down the user experience slightly, yet the experiment showed that users were clicking more! Why would that be? .... The "success" of getting users to click more was not real, but rather an instrumentation difference. Chrome, Firefox, and Safari are aggressive about terminating requests on navigation away from the current page and a non-negligible percentage of clickbeacons never make it to the server. This is especially true for the Safari browser, where losses are sometimes over 50%.

Primacy effect occurs when you change the navigation on a web site, and experienced users may be less efficient until they get used to the new navigation, thus giving an inherent advantage to the Control. Conversely, when a new design or feature is introduced, some users will investigate the new feature, click everywhere, and thus introduce a "novelty" bias that dies quickly if the feature is not truly useful.

For some metrics like Sessions/user, the confidence interval width does not change much over time. When looking for effects on such metrics, we must run the experiments with more users per day in the Treatment and Control.

The statistical theory of controlled experiments is well understood, but the devil is in the details and the difference between theory and practice is greater in practice than in theory ... It's easy to generate p-values and beautiful 3D graphs of trends over time. But the real challenge is in understanding when the results are invalid, not at the sixth decimal place, but before the decimal point, or even at the plus/minus for the percent effect ... Generating numbers is easy; generating numbers you should trust is hard!

Love the example of short-term metrics improving when they accidentally hurt search result quality (which caused people to click on ads rather than search results). That reminds me of a problem we had at Amazon where pop-up ads won A/B tests. Sadly, pop-up ads stayed up for months, until, eventually, we could show that they were hurting long-term customer happiness (and revenue) even if they showed higher revenue in the very short-term, and finally we were able to take pop-up ads down.

The whole paper is a great read. The authors have a lot of experience with A/B testing in practice and all the problems you encounter with A/B testing in practice. Definitely good to learn from their experience.

Read More

Quick links

What has caught my attention lately:

• The coming bandwidth explosion, by 2015, a zetabyte per year, "the equivalent of all movies ever made crossing IP networks every four minutes" ([1])


• Googlers (including gurus Jeff Dean and Andrew Ng) train "perhaps one of the largest known networks to date" on 1000 machines for three days, shows it is possible to learn to detect faces without ever labeling any of the data as a face, which has crazy cool implications for how the brain works ([1])


• John Dvorak says, "Windows 8 looks to me to be an unmitigated disaster".  Long-time Microsoft reporter Todd Bishop writes, "Microsoft [uses] 'fast and fluid' to describe Windows 8, but two other words keep popping to my mind: New Coke." A former Palm executive says, "To me, it feels like Microsoft is in a quiet panic. When Apple says the era of the PC has ended, I think Microsoft may believe it even more than Apple does." ([1] [2] [3])


• "Make everyone a manager" aka "why there are no bosses" ([1] [2] [3] [4] [5])


• Usability study shows people ignore social annotations when searching (e.g. one of your friends clicked or liked a web page that is in your search results) and, in the rare cases when searchers pay attention to them, people don't find them useful. Pretty serious implications for Facebook's partnership with Bing and Google's increasingly annoying habit of slopping Google+ all over Google web search. ([1]) 


• Prefetching entire web pages has issues, but a lot of the benefit can be gained just with DNS prefetching and TCP preconnecting.   "If we guess right, the DNS and TCP handshake may complete before we even hit enter."  Lots of clever ideas in Google Chrome, including this one. ([1])


• Chatbot for Facebook and Twitter that basically does Let Me Google That For You, very funny, but also surprisingly effective ([1])


• Retention, morale, productivity, and recruiting have long been wastelands of opinion over data, but here is a very welcome initiative to change that spearheaded by Google. By the way, I love that Google doesn't have Human Resources, they have "People Ops". ([1])


• The time has come to optimize code for energy consumed. Don't miss the table showing how many joules it takes to render the home pages of some popular websites. ([1])


• At least for ambiguous queries like [pizza], Google no longer emphasizes search results, not a bad idea, but quite a change for them ([1])


• Google is now crawling and executing most Javascript on the Web. This has been talked about for years, but first I've heard of it being done routinely at massive scale. ([1])


• For shopping search results, it matters what products you show next to each other.  In particular, people often buy in the middle of the price range, so the mix of prices on the products you show can change what product people buy. ([1])


• Is Google's mission now organize the world's information and make it universally accessible and useful (but only if you pay to be included)? ([1])


• Clever idea here, using playlists (or wishlists or any kind of publicly available list) for recommendations. ([1])


• A WWW 2012 paper that both quotes Xkcd and contains the sentence "this is quite a sweet ass-abstract for a scientific paper, dude!" in the introduction. Fun paper as well, but I think it's notable for that alone. ([1])


• Remarkable ability to automatically stabilize the camera in YouTube videos. Make sure to look at the two videos. ([1])


• Can't wait for us to have full wall displays controlled by voice and gesture, these are another step closer ([1] [2])


• Udacity and Coursera are getting more explicit about their business model, which appears to be sourcing programmers nicely pre-screened for coding ability to companies. ([1] [2])


• On flipped classrooms: "A control class that received a lecture from a Nobel Prize-winning physicist and an experimental section where students worked with graduate assistants to solve physics problems. Test scores for the experimental group (non-lecture) was nearly double that of the control section (41% to 74%)." ([1])


• Love that robot swarms are practical and starting to be used. Want to see this for space and undersea probes too. ([1])


• A big step toward computers that accept error as normal, yielding large gains in efficiency and performance ([1])


• Talk about geniuses at Facebook ignores the big problem that no one -- not Google, Yahoo, Microsoft, Facebook, or any of the newspapers -- knows how to solve this problem of making advertising relevant, effective, and lucrative without immediate purchase intent, despite years of work by thousands of brilliant people ([1] [2])


• "The buy, driven entirely by Zuckerberg, was made because Facebook’s CEO was petrified of Instagram becoming a Twitter-owned property. Zuckerberg, we’re told, lives in perpetual anxiety, preoccupied by the fear of Facebook losing its place ... That fear served as the catalyst behind his decision to buy Instagram." ([1] [2])


• Fascinating (but long) talk by Googler Steve Seitz on "the next generation of Google Maps". Also a great survey of work (older work and work currently at Google) using crowdsourced photos to build 3D walkthroughs of cities and interiors for photo tourism. Relevant for Apple launching their own maps, but also amazing and fun, can't wait to see more of this launched. ([1])


• The decline of TV has been greatly exaggerated ([1])


• Ubiquitous Wi-Fi might be back, this time with cell phone-like handoff between hotspots, could be a real game changer ([1])


• One of the better articles on this: "5 Things You Should Know Before Working at a Start-Up" ([1])


• "The average VC fund returns less money to investors than they invested in the first place." ([1] [2])


• Truly random (and fast) number generators by looking at quantum vacuum, possibly soon small and cheap enough to be put on a motherboard. Sounds trivial, I know, but insufficiently random numbers cause security and performance problems, so this really is a pretty useful idea. ([1])


• "Geek" has really changed meaning in the last decade, now almost admired. ([1])

Read More