Quick links

What has caught my attention lately:

• Videos showing Windows 8 is horribly painful for most people, looks likely to be another Windows Vista-like flop. Really worth watching the videos or trying it yourself (videos [1] [2], try it [1] [2])
  
  
• "38% of the ads are never in view to a user" and another 12% "of the ads are in view for less than 0.5 seconds" ([1])
  
  
• "Many more ads" are coming on Facebook, "a lot more advertising ... [on] Facebook's traditionally clean interface." Could this mean Facebook is having revenue trouble already? ([1] [2] [3])
  
  
• Not only is the iPhone over half of Apple's revenue, it is more than 70% of their profits. Apple really is a mobile phone manufacturer with a few other businesses attached. ([1])
  
  
• Coming soon, a "voice-activated assistant that remembers everything you say ... systems that are more conversational, that have the ability to ask more sophisticated followup questions and adapt to the individual ... [with] short-term and long-term memory." ([1])
  
  
• "Microsoft tries to find pockets of unrealized revenue and then figures out what to make. Apple is just the opposite: It thinks of great products, then sells them." ([1])
  
  
• "The best way to get the most out of engineers is to surround them with other great engineers." ([1])
  
  
• "It’s positively de-motivating to work for a company where your job is just to shut up and take orders. In tech startup land, we all understand instinctively that we have to hire super smart people, but we forget that we then have to organize the workforce so that those people can use their brains." ([1])
  
  
• Programmers want to learn new skills and technology while working in a team of people they respect, and over 90% of programmers said they are willing to take a lower paying job to get that. ([1])
  
  
• Netflix's streaming catalog continues to deteriorate, is now down to only 853 good movies, of which only 155 were released within the last five years ([1] [2] [3])
  
  
• "This class is about setting you on the path to developing good taste as a programmer" (free, from Udacity, taught by Googler and AI guru Peter Norvig, starts Apr 16) ([1])
  
  
• Could this be the business model for Udacity? Offer free classes online, then send companies candidates pre-screened for machine learning programming ability? ([1] [2])
  
  
• If my blog is any indication, the only RSS feeder still being used is Google Reader. Are all others dead now? ([1])
  
  
• Huge and wide open opportunity in personalized advertising for online news. Amazes me Yahoo and Amazon haven't gone after this, and that Google hasn't done a better job going after it. ([1])
  
  
• Paper with fascinating statistics on Groupon and other daily deal sites. Most dramatic, it costs restaurants half a star in their Yelp rating if they offer Groupon deals. ([1])

Read More

Ad targeting at Yahoo

A remarkably detailed paper, "Web-Scale User Modeling for Targeting" (PDF), will be presented at WWW 2012 that gives many insights into how Yahoo does personalized advertising.

In summary, the researchers describe a system used in production at Yahoo that does daily builds of large user profiles. Each profile contains tens of thousands of features that summarize the interests of each user from the web pages they have viewed, searches they made, and ads they have viewed, clicked on, and converted (bought something) on. They explain how important it is to use conversions, not just ad clicks, to train the system. They measure the importance of using recent history (what you did in the last couple days), of using fine-grained data (detailed categories and even some specific pages and queries), of using large profiles, and of including data about ad views (which is a huge and low quality data source since there are multiple ad views per page view), and find all those significantly help performance.

Some excerpts from the paper:

We present the experiences from building a web-scale user modeling platform for optimizing display advertising targeting at Yahoo .... Our work ... [looks] into understanding the effect of different user activities on prediction, [gives] insights about the temporal aspect of user behavior (recency vs. long-term trends), and [explores] different variants (user representation and target label) through large offline and online experiments .... We deployed our platform to production and achieved a [large] boost in online metrics, such as eCPA, compared to the old system.

Our objective is to refine the targeting constraints using the past behavior of the users ... [so] we can improve the number of conversions per ad impression without greatly increasing the number of impressions.

User profiles are aggregated logs from different systems/products (e.g. user logs of Yahoo News, Yahoo Finance, etc.) .... We consider several different events ... [including] pages visited .. the category of the page ... searches issued, clicks on search links, clicks on search advertising links ... [and] the category of the search query ... [and] views and clicks on ads ... [and] the ad category ... from an existing hierarchical ad categorizer.

Our results show [a] large performance loss incurred in favoring long-term history over short-term history. This is obvious as the recent history clearly communicates with a high probability the current interest of the user ... Although recent history is more important than older history, we still need to include older history to get the most complete idea about the user.

Results show ... many of our raw features are completely non-discriminative. However, a small percentage of these features are actually important ... [For example, just] ... dropping all raw ad views ... [or if] we drop all raw features and only keep categorical features ... [causes] dropping [of] the weighted AUC measure by 3.69% and 4.26%, respectively ... In production ... we apply a coarse feature selection through mutual information, then we apply a rigorous feature selection through l₁ regularization.

Very interesting. A couple things I am left wondering:

First, they found recent history is very effective, yet only update the profiles daily. Wouldn't their results on the value of recent behavior (which others found too) suggest that there would be benefit from hourly or, even better, real-time updates of the profiles (perhaps with a second memory-based, unreliable, and partial coverage system supplementing the data in the more complete and more accurate older profiles)? That would allow the system to adapt immediately when someone, for example, starts looking at information for a vacation to Hawaii and show relevant offers immediately instead of only being able to do it the next day when it is usually too late. Unfortunately, I suspect we're not going to see really big gains in relevance and usefulness of ads without real-time updates to profiles of fine-grained interests; results that show that data only 24 hours old is better than data a week old may only be a tease of the gains to be seen with data only seconds old.

Second, they find that features based on individual search queries and pages viewed ("raw features") usually have no value, but occasionally have enough value that it is important to include some. Wouldn't that suggest that the categorization scheme for pages viewed, searches made, and ads need to be more fine-grained (e.g. not just the category "pants", but the category "men's boot cut jeans")? Or, better, perhaps more fine-grained while also correctly cross correlated (interest in "men's boot cut jeans" not only shows in the data a weak interest in all pants, but also maybe has been shown to indicate a fairly strong interest in "men's flannel shirts")?

If you are interested in this paper, you might also want to look at another recent paper out of Yahoo Research, "Learning to Target: What Works for Behavioral Advertising" (ACM), which is referenced multiple times by this paper and describes the features used in the user profile in a bit more detail, as well as the results of some other experiments.

Please see also my 2007 post, "What to advertise when there is no commercial intent?"

Read More

More quick links

More of what has caught my attention lately:

• Laptops with Kinect sensors are coming. Worth paying attention to, gesturing in air to issue commands, a very different UX could be built on top of this ([1] [2])
  
  
• "Each streaming subscriber is worth only $2.40 in profit each quarter to Netflix, compared to $17.32 for each DVD subscriber. The old business was very lucrative. The new business kind of sucks." ([1])
  
  
• "You're not going to get content owners to license ... for less than what they get from the cable companies ... [if you will] use that cheap content to destroy the cable companies' business model." ([1])
  
  
• "Federal officials approached Google with evidence of its employees' wrongdoing ... Google agreed to pay $500 million to ... ward off criminal charges against the company." ([1])
  
  
• Google is spending nearly $1B every quarter buying new servers and data centers. That buys a lot of machines. ([1] [2])
  
  
• Education startups are suddenly very, very hot. ([1] [2] [3] [4])
  
  
• "Tiny directional antennas at the top of each rack ... send and receive data. A central controller monitors traffic patterns, finds network bottlenecks, configures the antennas and turns on the wireless links when more bandwidth is required ... The design sped up traffic by at least 45 percent." ([1])
  
  
• "Wimpy cores are fine, but if you go down to the wimpiest range, your gains really have to be enormous if you want to consider all the aggravation -- and the hit to their productivity -- that your software engineers face." ([1])
  
  
• A Facebook engineer explains why is actually the right thing for Facebook to produce buggy code ([1])
  
  
• "How sex, bombs, and burgers shaped our world" ([1])
  
  
• "There is a monolithic view that this generation of technology I.P.O.'s is completely broken." ([1])
  
  
• Just three engineers built and run Instagram, which has 14 million users, 150 million photos, several terabytes of data, and hundreds of machines. ([1] [2])
  
  
• Startup founders "say that if they'd known when they were starting their company about the obstacles they'd have to overcome, they might never have started it." ([1])
  
  
• Two 17-year-olds used a weather balloon to send a little Lego astronaut and a video camera 15 miles into the stratosphere. Very fun. ([1])

Read More

Quick links

Some of what has caught my attention recently:

• Security guru Bruce Schneier predicts "smart phones are going to become the primary platform of attack for cybercriminals" soon ([1])
  
  
• If, next, Amazon does a smartphone, I hope it is WiFi-based, like Steve Jobs originally wanted to do with the iPhone ([1] [2] [3])
  
  
• iPhone owners love Siri despite its flaws ([1])
  
  
• Valve, makers of Steam, talks about their pricing experiments: "Without making announcements, we varied the price ... pricing was perfectly elastic ... Then we did this different experiment where we did a sale ... a highly promoted event ... a 75 percent price reduction ... gross revenue [should] remain constant. Instead what we saw was our gross revenue increased by a factor of 40. Not 40 percent, but a factor of 40 ... completely not predicted by our previous experience with silent price variation." [[1]]
  
  
• An idea whose time has come, profiling code based not on the execution time required, but the power consumed ([1])
  
  
• Grumpy about work and dreaming about doing a startup? Some food for thought for those romanticizing startup life. ([1] [2])
  
  
• Yahoo discovers toolbar data (the urls people click on and browse to) helps a lot for web crawling ([1])
  
  
• Google Personalized Search adds explanations. Explanations not only add credibility to recommendations, but also make people more accepting of recommendations they don't like. ([1])
  
  
• "Until now, many education studies have been based on populations of a few dozen students. Online technology can capture every click: what students watched more than once, where they paused, what mistakes they made ... [massive] data ... for understanding the learning process and figuring out which strategies really serve students best." ([1])
  
  
• Andrew Ng's machine learning class at Stanford was excellent; I highly recommend it. If you missed it the first time, it is being offered again (for free again) next quarter. ([1])
  
  
• Microsoft giving up on its version of Hadoop? Surprising. ([1])
  
  
• The NYT did a fun experiment crowdsourcing predictions. The results are worth a look. ([1] [2])
  
  
• Web browsers (Firefox and Chrome) will be a gaming platform soon ([1] [2])
  

Read More

Browsing behavior for web crawling

A recent paper out of Yahoo, "Discovering URLs through User Feedback" (ACM), describes the value from using what pages people browse to and click on (which is in Yahoo's toolbar logs) to inform their web crawler about new pages to crawl and index.

From the paper:

Major commercial search engines provide a toolbar software that can be deployed on users' Web browsers. These toolbars provide additional functionality to users, such as quick search option, shortcuts to popular sites, and malware detection. However, from the perspective of the search engine companies, their main use is on branding and collecting marketing statistics. A typical toolbar tracks some of the actions that the user performs on the browser (e.g., typing a URL, clicking on a link) and reports these actions to the search engine, where they are stored in a log file.

A Web crawler continuously discovers new URLs and fetches their content ... to build an inverted index to serve [search] queries. Even though the basic mechanism of a crawler is simple, crawling efficiently and eff ectively is a difficult problem ... The crawler not only has to continuously enlarge its repository by expanding its frontier, but also needs to refresh previously fetched pages to incorporate in its index the changes on those pages. In practice, crawlers prioritize the pages to be fetched, taking into account various constraints: available network bandwidth, peak processing capacity of the backend system, and politeness constraints of Web servers ... The delay to discover a Web page can be quite long after its creation and some Web sites may be only partially crawled. Another important challenge is the discovery of hidden Web content ... often ... backed by a database.

Our work is the first to evaluate the benefits of using the URLs collected from a Web browser toolbar as a form of user feedback to the crawling process .... On average, URLs accessed by the users are more important than those found ... [by] the crawler ... The crawler has a significant delay in discovering URLs that are first accessed by the users ... Finally, we [show] that URL discovery via toolbar [has a] positive impact on search result quality, especially for queries seeking recently created content and tail content.

The paper goes on to quantify the surprisingly large number of URLs found by the toolbar that are useful, not private, and not excluded by robots.txt. Importantly, a lot of these are deep web pages, only visible by doing a query on a database, and hard to ferret out of that database any way but looking at the pages people actually look at.

Also interesting are the metrics on pages the toolbar data finds first. People often send links to new web pages by e-mail or text message. Eventually, those links might appear on the web, but eventually can be a long time, and many of the urls found first in the toolbar data ("more than 60%") are found way before the crawler manages to discover them ("at least 90 days earlier than the crawler").

Great paper out of Yahoo Research and a great example of how useful behavior data can be. It is using big data to help people help others find what they found.

Read More

What mobile location data looks like to Google

A recent paper out of Google, "Extracting Patterns From Location History" (PDF), is interesting not only for confirming that Google is studying using location data from mobile devices for a variety of purposes, but also for the description of the data they can get.

From the paper:

Google Latitude periodically sends his location to a server which shares it with his registered friends.

A user's location history can be used to provide several useful services. We can cluster the points to determine where he frequents and how much time he spends at each place. We can determine the common routes the user drives on, for instance, his daily commute to work. This analysis can be used to provide useful services to the user. For instance, one can use real-time traffic services to alert the user when there is traffic on the route he is expected to take and suggest an alternate route.

Much previous work assumes clean location data sampled at very high frequency ... [such as] one GPS reading per second. This is impractical with today's mobile devices due to battery usage ... [Inferring] locations by listening to RF-emissions from known wi-fi access points ... requires less power than GPS ... Real-world data ... [also] often has missing and noisy data.

17% of our data points are from GPS and these have an accuracy in the 10 meter range. Points derived from wifi signatures have an accuracy in the 100 meter range and represent 57% of our data. The remaining 26% of our points are derived from cell tower triangulation and these have an accuracy in the 1000 meter range.

The paper goes on to describe how they clean the data and pin noisy location trails to roads. But the most interesting tidbit for me was how few of their data points come from GPS and how much they have to rely on less accurate cell tower and WiFi hotspot triangulation.

A lot of people have assumed mobile devices would provide nice trails of accurate and frequently sampled locations. But, if the Googlers' data is typical, it sounds like location data from mobile devices is going to be very noisy and very sparse for a long time.

Read More

Even more quick links

Even more of what has caught my attention recently:

• Spooky but cool research: "Electrical pulses to the brain and muscles ... activate and deactivate the insect's flying mechanism, causing it to take off and land ... Stimulating certain muscles behind the wings ... cause the beetle to turn left or right on command." ([1])
  
  
• Good rant: "Our hands feel things, and our hands manipulate things. Why aim for anything less than a dynamic medium that we can see, feel, and manipulate? ... Pictures Under Glass is old news ... Do you seriously think the Future Of Interaction should be a single finger?" ([1])
  
  
• Googler absolutely shreds traditional Q&A and argues that the important thing is getting a good product, not implementing a bad product correctly to spec. Long talk, if you're short on time, the talk starts at 6:00, meat of the talk starts at 13:00, and the don't miss parts of the talk are at 17:00 and 21:00. ([1])
  
  
• "There has been very little demand for Chromebooks since Acer and Samsung launched their versions back in June. The former company reportedly only sold 5,000 units by the end of July, and the latter Samsung was said to have sold even less than that in the same timeframe." ([1])
  
  
• With the price change to offer Kindles at $79, Amazon is now selling them below cost ([1])
  
  
• Personalization applied to education, using the "combined data power of millions of students to provide uniquely personalized learning to each." ([1] [2] [3] [4] [5] [6])
  
  
• It is common to use human intuition to choose algorithms and tune parameters on algorithms, but this is the first I've ever heard of using games to crowdsource algorithm design and tuning ([1])
  
  
• Great slides from a Recsys tutorial by Daniel Tunkelang, really captures the importance of UX and HCIR in building recommendation and personalization features ([1])
  
  
• Bing finally figured out that when judges disagree with clicks, clicks are probably right ([1])
  
  
• Easy to forget, but the vast majority of US mobile devices still are dumbphones ([1])
  
  
• Finally, finally, Microsoft produces a decent mobile phone ([1])
  
  
• Who needs a touch screen when any surface can be a touch interface? ([1])
  
  
• Impressive augmented reality research demo using Microsoft Kinect technology ([1])
  
  
• Very impressive new technique for adding objects to photographs, reproducing lighting, shadows, and reflections, and requiring just a few corrections and hints from a human about the geometry of the room. About as magical as the new technology for reversing camera shake to restore out-of-focus pictures to focus. ([1] [2])
  
  
• Isolation isn't complete in the cloud -- your neighbors can hurt you by hammering the disk or network -- and some startups have decided to go old school back to owning the hardware ([1] [2])
  
  
• "The one thing that Siri cannot do, apparently, is converse with Scottish people." ([1])
  
  
• Amazon grew from under 25,000 employees to over 50,000 in two years ([1])
  
  
• Google Chrome is pushing Mozilla into bed with Microsoft? Really? ([1])
  
  
• Is advice Steve Jobs gave to Larry Page the reason Google is killing so many products lately? ([1])
  
  
• Why does almost everyone use the default software settings? Research says it appears to be a combination of minimizing effort, an assumption of implied endorsement, and (bizarrely) loss aversion. ([1])

Read More