developers

Showing 13 posts tagged developers

Dash Open Podcast: Episode 02 - Building Community and Mentorship around Hackdays

By Ashley Wolf, Open Source Program Manager, Verizon Media

The second installment of Dash Open is ready for you to tune in!

In this episode, Gil Yehuda, Sr. Director of Open Source at Verizon Media, interviews Dav Glass, Distinguished Architect of IaaS and Node.js at Verizon Media. Dav discusses how open source inspired him to start HackSI, a Hack Day for all ages, as well as robotics mentorship programs for the Southern Illinois engineering community.

Listen now on iTunes or SoundCloud.

Dash Open is your place for interesting conversations about open source and other technologies, from the open source program office at Verizon Media. Verizon Media is the home of many leading brands including Yahoo, Aol, Tumblr, TechCrunch, and many more.

Follow us on Twitter @YDN and on LinkedIn.

Achieving Major Stability and Performance Improvements in Yahoo Mail with a Novel Redux Architecture

By Mohit Goenka, Gnanavel Shanmugam, and Lance Welsh

At Yahoo Mail, we’re constantly striving to upgrade our product experience. We do this not only by adding new features based on our members’ feedback, but also by providing the best technical solutions to power the most engaging experiences. As such, we’ve recently introduced a number of novel and unique revisions to the way in which we use Redux that have resulted in significant stability and performance improvements. Developers may find our methods useful in achieving similar results in their apps.

Improvements to product metrics

Last year Yahoo Mail implemented a brand new architecture using Redux. Since then, we have transformed the overall architecture to reduce latencies in various operations, reduce JavaScript exceptions, and better synchronized states. As a result, the product is much faster and more stable.

Stability improvements:

  • when checking for new emails – 20%
  • when reading emails – 30%
  • when sending emails – 20%

Performance improvements:

  • 10% improvement in page load performance
  • 40% improvement in frame rendering time

We have also reduced API calls by approximately 20%.

How we use Redux in Yahoo Mail

Redux architecture is reliant on one large store that represents the application state. In a Redux cycle, action creators dispatch actions to change the state of the store. React Components then respond to those state changes. We’ve made some modifications on top of this architecture that are atypical in the React-Redux community.

For instance, when fetching data over the network, the traditional methodology is to use Thunk middleware. Yahoo Mail fetches data over the network from our API. Thunks would create an unnecessary and undesirable dependency between the action creators and our API. If and when the API changes, the action creators must then also change. To keep these concerns separate we dispatch the action payload from the action creator to store them in the Redux state for later processing by “action syncers”. Action syncers use the payload information from the store to make requests to the API and process responses. In other words, the action syncers form an API layer by interacting with the store. An additional benefit to keeping the concerns separate is that the API layer can change as the backend changes, thereby preventing such changes from bubbling back up into the action creators and components. This also allowed us to optimize the API calls by batching, deduping, and processing the requests only when the network is available. We applied similar strategies for handling other side effects like route handling and instrumentation. Overall, action syncers helped us to reduce our API calls by ~20% and bring down API errors by 20-30%.

Another change to the normal Redux architecture was made to avoid unnecessary props. The React-Redux community has learned to avoid passing unnecessary props from high-level components through multiple layers down to lower-level components (prop drilling) for rendering. We have introduced action enhancers middleware to avoid passing additional unnecessary props that are purely used when dispatching actions. Action enhancers add data to the action payload so that data does not have to come from the component when dispatching the action. This avoids the component from having to receive that data through props and has improved frame rendering by ~40%. The use of action enhancers also avoids writing utility functions to add commonly-used data to each action from action creators.

image

In our new architecture, the store reducers accept the dispatched action via action enhancers to update the state. The store then updates the UI, completing the action cycle. Action syncers then initiate the call to the backend APIs to synchronize local changes.

Conclusion

Our novel use of Redux in Yahoo Mail has led to significant user-facing benefits through a more performant application. It has also reduced development cycles for new features due to its simplified architecture. We’re excited to share our work with the community and would love to hear from anyone interested in learning more.

How to Make Your Web App More Reliable and Performant Using webpack: a Yahoo Mail Case Study

image

By Murali Krishna Bachhu, Anurag Damle, and Vishal Patel

As engineers on the Yahoo Mail team at Oath, we pride ourselves on the things that matter most to developers: faster development cycles, more reliability, and better performance. Users don’t necessarily see these elements, but they certainly feel the difference they make when significant improvements are made. Recently, we were able to upgrade all three of these areas at scale by adopting webpack® as Yahoo Mail’s underlying module bundler, and you can do the same for your web application.

What is webpack?

webpack is an open source module bundler for modern JavaScript applications. When webpack processes your application, it recursively builds a dependency graph that includes every module your application needs. Then it packages all of those modules into a small number of bundles, often only one, to be loaded by the browser.

webpack became our choice module bundler not only because it supports on-demand loading, multiple bundle generation, and has a relatively low runtime overhead, but also because it is better suited for web platforms and NodeJS apps and has great community support.

image
Comparison of webpack to other open source bundlers

How did we integrate webpack?

Like any developer does when integrating a new module bundler, we started integrating webpack into Yahoo Mail by looking at its basic config file. We explored available default webpack plugins as well as third-party webpack plugins and then picked the plugins most suitable for our application. If we didn’t find a plugin that suited a specific need, we wrote the webpack plugin ourselves (e.g., We wrote a plugin to execute Atomic CSS scripts in the latest Yahoo Mail experience in order to decrease our overall CSS payload**).

During the development process for Yahoo Mail, we needed a way to make sure webpack would continuously run in the background. To make this happen, we decided to use the task runner Grunt. Not only does Grunt keep the connection to webpack alive, but it also gives us the ability to pass different parameters to the webpack config file based on the given environment. Some examples of these parameters are source map options, enabling HMR, and uglification.

Before deployment to production, we wanted to optimize the javascript bundles for size to make the Yahoo Mail experience faster. webpack provides good default support for this with the UglifyJS plugin. Although the default options are conservative, they give us the ability to configure the options. Once we modified the options to our specifications, we saved approximately 10KB.

image
Code snippet showing the configuration options for the UglifyJS plugin

Faster development cycles for developers

While developing a new feature, engineers ideally want to see their code changes reflected on their web app instantaneously. This allows them to maintain their train of thought and eventually results in more productivity. Before we implemented webpack, it took us around 30 seconds to 1 minute for changes to reflect on our Yahoo Mail development environment. webpack helped us reduce the wait time to 5 seconds.

More reliability

Consumers love a reliable product, where all the features work seamlessly every time. Before we began using webpack, we were generating javascript bundles on demand or during run-time, which meant the product was more prone to exceptions or failures while fetching the javascript bundles. With webpack, we now generate all the bundles during build time, which means that all the bundles are available whenever consumers access Yahoo Mail. This results in significantly fewer exceptions and failures and a better experience overall.

Better Performance

We were able to attain a significant reduction of payload after adopting webpack.

  1. Reduction of about 75 KB gzipped Javascript payload
  2. 50% reduction on server-side render time
  3. 10% improvement in Yahoo Mail’s launch performance metrics, as measured by render time above the fold (e.g., Time to load contents of an email).

Below are some charts that demonstrate the payload size of Yahoo Mail before and after implementing webpack.

image
Payload before using webpack (JavaScript Size = 741.41KB)

image
Payload after switching to webpack (JavaScript size = 669.08KB)

image

Conclusion

Shifting to webpack has resulted in significant improvements. We saw a common build process go from 30 seconds to 5 seconds, large JavaScript bundle size reductions, and a halving in server-side rendering time. In addition to these benefits, our engineers have found the community support for webpack to have been impressive as well. webpack has made the development of Yahoo Mail more efficient and enhanced the product for users. We believe you can use it to achieve similar results for your web application as well.

**Optimized CSS generation with Atomizer

Before we implemented webpack into the development of Yahoo Mail, we looked into how we could decrease our CSS payload. To achieve this, we developed an in-house solution for writing modular and scoped CSS in React. Our solution is similar to the Atomizer library, and our CSS is written in JavaScript like the example below:

image
Sample snippet of CSS written with Atomizer

Every React component creates its own styles.js file with required style definitions. React-Atomic-CSS converts these files into unique class definitions. Our total CSS payload after implementing our solution equaled all the unique style definitions in our code, or only 83KB (21KB gzipped).

During our migration to webpack, we created a custom plugin and loader to parse these files and extract the unique style definitions from all of our CSS files. Since this process is tied to bundling, only CSS files that are part of the dependency chain are included in the final CSS.

Open Sourcing Vespa, Yahoo’s Big Data Processing and Serving Engine

image



By Jon Bratseth, Distinguished Architect, Vespa

Ever since we open sourced Hadoop in 2006, Yahoo – and now, Oath – has been committed to opening up its big data infrastructure to the larger developer community. Today, we are taking another major step in this direction by making Vespa, Yahoo’s big data processing and serving engine, available as open source on GitHub.

image
Vespa architecture overview

Building applications increasingly means dealing with huge amounts of data. While developers can use the Hadoop stack to store and batch process big data, and Storm to stream-process data, these technologies do not help with serving results to end users. Serving is challenging at large scale, especially when it is necessary to make computations quickly over data while a user is waiting, as with applications that feature search, recommendation, and personalization.

By releasing Vespa, we are making it easy for anyone to build applications that can compute responses to user requests, over large datasets, at real time and at internet scale – capabilities that up until now, have been within reach of only a few large companies.

Serving often involves more than looking up items by ID or computing a few numbers from a model. Many applications need to compute over large datasets at serving time. Two well-known examples are search and recommendation. To deliver a search result or a list of recommended articles to a user, you need to find all the items matching the query, determine how good each item is for the particular request using a relevance/recommendation model, organize the matches to remove duplicates, add navigation aids, and then return a response to the user. As these computations depend on features of the request, such as the user’s query or interests, it won’t do to compute the result upfront. It must be done at serving time, and since a user is waiting, it has to be done fast. Combining speedy completion of the aforementioned operations with the ability to perform them over large amounts of data requires a lot of infrastructure – distributed algorithms, data distribution and management, efficient data structures and memory management, and more. This is what Vespa provides in a neatly-packaged and easy to use engine.

With over 1 billion users, we currently use Vespa across many different Oath brands – including Yahoo.com, Yahoo News, Yahoo Sports, Yahoo Finance, Yahoo Gemini, Flickr, and others – to process and serve billions of daily requests over billions of documents while responding to search queries, making recommendations, and providing personalized content and advertisements, to name just a few use cases. In fact, Vespa processes and serves content and ads almost 90,000 times every second with latencies in the tens of milliseconds. On Flickr alone, Vespa performs keyword and image searches on the scale of a few hundred queries per second on tens of billions of images. Additionally, Vespa makes direct contributions to our company’s revenue stream by serving over 3 billion native ad requests per day via Yahoo Gemini, at a peak of 140k requests per second (per Oath internal data).

With Vespa, our teams build applications that:

  • Select content items using SQL-like queries and text search
  • Organize all matches to generate data-driven pages
  • Rank matches by handwritten or machine-learned relevance models
  • Serve results with response times in the low milliseconds
  • Write data in real-time, thousands of times per second per node
  • Grow, shrink, and re-configure clusters while serving and writing data

To achieve both speed and scale, Vespa distributes data and computation over many machines without any single master as a bottleneck. Where conventional applications work by pulling data into a stateless tier for processing, Vespa instead pushes computations to the data. This involves managing clusters of nodes with background redistribution of data in case of machine failures or the addition of new capacity, implementing distributed low latency query and processing algorithms, handling distributed data consistency, and a lot more. It’s a ton of hard work!

As the team behind Vespa, we have been working on developing search and serving capabilities ever since building alltheweb.com, which was later acquired by Yahoo. Over the last couple of years we have rewritten most of the engine from scratch to incorporate our experience onto a modern technology stack. Vespa is larger in scope and lines of code than any open source project we’ve ever released. Now that this has been battle-proven on Yahoo’s largest and most critical systems, we are pleased to release it to the world.

Vespa gives application developers the ability to feed data and models of any size to the serving system and make the final computations at request time. This often produces a better user experience at lower cost (for buying and running hardware) and complexity compared to pre-computing answers to requests. Furthermore it allows developers to work in a more interactive way where they navigate and interact with complex calculations in real time, rather than having to start offline jobs and check the results later.

Vespa can be run on premises or in the cloud. We provide both Docker images and rpm packages for Vespa, as well as guides for running them both on your own laptop or as an AWS cluster.

We’ll follow up this initial announcement with a series of posts on our blog showing how to build a real-world application with Vespa, but you can get started right now by following the getting started guide in our comprehensive documentation.

Managing distributed systems is not easy. We have worked hard to make it easy to develop and operate applications on Vespa so that you can focus on creating features that make use of the ability to compute over large datasets in real time, rather than the details of managing clusters and data. You should be able to get an application up and running in less than ten minutes by following the documentation.

We can’t wait to see what you’ll build with it!

Introducing: πŸ•πŸΊπŸ“± Meetups

By: Shirin Sabahi, Developer Relations

We’re excited to introduce….  

image

Originally posted by emmawearsburberry

Pizza Beer Mobile is our new meetup group, launching at our Yahoo! offices in NYC (Times Square) on June 2nd discussing How to Build a Bulletproof SDK.  

Flurry Analytics has been with you since the start of the mobile revolution and we have a ton of valuable insights we’d love to share to help you grow, retain and monetize your apps. Join us to drink beer, eat pizza and chat with other mobile-minded folks.

We’ll be continually updating our list of upcoming gatherings on the events tab found at developer.yahoo.com, so stay tuned for an event happening in your city.

Long live🍕🍺📱!

Yahoo’s NYC Hackathon Today!

By Jarah Euston, VP of Product

The Yahoo Mobile Developer Conference: NYC starts tomorrow, and we’re excited to kick things off with a hands-on Hackathon, where mobile developers will work on-site with Yahoo engineers to learn about the fully-loaded and recently upgraded Yahoo Mobile Developer Suite. Participants in the Hackathon will gain experience using the Suite to enhance, distribute and monetize their app content while building new apps and receiving real time insight and mentorship from our mobile experts. We will educate attendees about the opportunities they have to improve apps with more seamless user experiences by innovating with native ads, by incorporating Yahoo Search into their apps or by using the Tumblr API to keep users in their app.

image

The Hackathon runs from 9 a.m. until 6 p.m., when attendees will enter their hacks into our contest, and we will judge who has used the Suite to create the three most impressive demos using native ads, Yahoo Search syndication or the Tumblr API. Read more about the Hackathon and the Conference here: ymdcnyc.tumblr.com.

image

Hackathon winners will be announced tomorrow at YMDC and will take home some cool prizes.

Navigating Native: Mobile Monetization + Pizza in SF next Thursday!

By Jarah Euston, Sr. Director of Marketing & Analytics, Flurry from Yahoo

It’s that time again. We’re gearing up for our third Yodel Meetup in our San Francisco office at 110 5th Street. Come by after work next Thursday, May 14 at 6pm to listen to a panel of experts talk mobile app monetization. We’ll enjoy beer and pizza and dive into the impact of native ads on app monetization, best practices and ad integration in apps. 

See details of the event below, and RSVP HERE.

image

Here’s some pics from last time. We can’t wait to see you!

image
image

Question? Want Yodel Meetups in your city? Yodel at us at emailing:marketing@flurry.com

Yahoo Mobile Developer Suite Now Includes Apple Watch Analytics

By Brad Jones, Director, Product Management, Flurry from Yahoo 

Today, we announced Apple Watch analytics within Flurry Analytics! With Flurry’s new Watch analytics, developers can understand user behavior in their apps across phones, tablets, and Watch extensions.

image

Developers have the ability to track custom Apple Watch events to measure how users are interacting with the experiences they develop for the smallest screen. They will also be able to understand how users engage with the app running on the Apple Watch and its parent app on the user’s smartphone.

Watch Metrics available are:

  • New Users
  • Active Users
  • Total Event Occurrences: Sum of the Total Events by Day
  • Daily Average Event Occurrences: This value is the sum of the Total Events by Day metric for the date range selected divided by the number of days in that period.
  • Daily Average Unique Event Users:  This value is the sum of the Unique Users Performing Event by Day metric for the date range selected divided by the number of days in that period.
  • Events Users as percent of Active App Users: Percentage of app users engaging with watch extension. This metric helps you understand how engagement balances out between the watch app and the phone app.

To get started, all developers need to do is create a Watch Extension Key in their iPhone app, get the latest SDK, add the Watch extension key to their app, and then submit it to the App Store. Developers can create a Watch extension key on the App Info page under the Manage menu when viewing their parent app’s metrics on dev.flurry.com. 

Get Started Now!

Video App Install Ads Now Available to Developers Across the Globe

By Jarah Euston, Sr. Dir. of Marketing and Analytics, Flurry from Yahoo

I’m excited to announce that app marketers and developers can now access video app install ads on Yahoo apps and across thousands of apps through Yahoo App Marketing. Research shows that consumers who download an app after watching a trailer use the apps 40 percent more often and 20 percent longer each and every time compared with regular display install ads.

Developers have been seeing success with video app install ads, “Flurry, and now Yahoo, video app install ads have brought us high quality installs and highly engaged customers,” said John Dionisio, Sr. Director Performance Marketing at Zynga. “We rely on them for both new game launches and sustained, profitable install campaigns.”

Typically, App marketing video ads see an 89% completion rate, and users acquired have up to 43% more app sessions than average. Now, through audience targeting available with Yahoo Gemini video app install ads, including the recent integration of Flurry Personas, marketers can reach users most likely to download their app. Yahoo has made it easier than ever to create video app install ads by adding it into Yahoo Gemini’s self-serve UI.

To start using video app install ads, go to gemini.yahoo.com today, or for more information on Yahoo App Marketing, visit developer.yahoo.com/advertise/.