Team Data Science

The Fix Is In

photo: JD Hancock

Big data are a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. The challenges include capture, curation, storage, search, sharing, transfer, analysis, and visualization.

So say the opening sentences of the “Big data” article in Wikipedia. The people primarily responsible for conquering those challenges are data scientists. Being a data scientist these days is rather like being a Renaissance person; one must possess knowledge of and competency in a wide variety of subjects related directly and indirectly to the fields of mathematics and science.

Fortunately, data science has a number of sub-specialties to share the load. Understanding–defining–who does what (capturing, curating, storing, searching, sharing, transferring, analyzing, visualizing, processing) and why they do it means companies building data science teams can intelligently choose the areas of specialization that will best serve their goals.

Five Roles You Need on Your Big Data Team

Of course there’s the data scientist, the coveted knight in shining armor who visualizes models and creates (and continuously optimizes) sophisticated algorithms to transform data into something useful. But she could not do her part to fulfill corporate expectations without the support of equally coveted

  • Data hygienists, who deal with the “dirty data” problems inherent in collecting data so the data is clean now and stays clean in future.
  • Data explorers, who burrow into all the data a company collects to determine what, if anything, can be done with it, including how data originally collected for a different reason might be repurposed.
  • Business solution architects, who structure and organize data so it’s properly updated and where it needs to be within the necessary timeframe of every query–a critical feature of today’s data science when queries are ‘answered’ in real-time.
  • Campaign experts, who, with an in-depth understanding of both the technology and marketing, can turn the knowledge derived from data into insight and then into advice.

Assembling a powerful data science team, whether that team is internal or third-party, is necessary to applying big data tools. However, success rests as heavily in the hands of the right corporate culture as it does in the right specialized people. The best solution? Welcome reevaluation, innovation, experimentation, and keep the focus on the end game.

10 Qualities a Data-Friendly Business Culture Needs

[ New Perspective ] Tokyo Metropolitan Government Building, Shinjuku, Tokyo, JapanUsing and continuously optimizing data with the long-term goal of providing customer value and increasing conversion sounds easy; it can be difficult to execute. Getting the most from your data requires hard work and a willingness to adapt, to experiment, to learn from mistakes, to correct those mistakes as quickly as possible and to keep doing the process over and over without end. But, no matter the scale, crunching data to generate information and insight is valuable only if the organizational structures are in place to support the effort.

One of the most important problems corporations face is creating, at all levels of leadership a framework of support for and participation in new technologies that can help refine and expand how they do business. As pressure to adopt big data technologies grows stronger, many organizations have understandable fears concerning the integrity of their businesses. At the very least, leadership often perceives these technologies as threats to institutional knowledge and continuity.

Alex Miller of QVC, Anthony Bucci of RevZilla and Slava Sambu of Office Max have offered insight into the benefits of staying current with data processing technologies. They have discussed some of the stumbling blocks they have faced. But, as these three suggest, stumbling blocks are not dead ends. Organizations able to confront problems toward finding solutions soon begin to reap the rewards associated with using data effectively.

One of the goals—and it will become increasingly important—is corporate agility. With the ability to analyze and operate in real-time, a business can be far more responsive to its customers. It can quickly evaluate what is or is not working and correct the problem. It can identify problems and fix them immediately. For example, some data a business collects is going to be “dirty.” This is, and has always been, a common problem. Implementation of information derived from problematic data can potentially create a large problem with a large impact on the bottom line. A company needs to be agile enough to catch these problems, evaluate their nature and devise appropriate solutions as quickly as possible. As in right then; better still, an hour ago. Waiting while information makes its way slowly up the chain as each management level makes a decision—even waiting as long as it takes to organize a meeting of department heads—is not an option anymore.

Businesses are accomplishing this every day, and some have been employing these practices for years, decades. Amazon operates like this. Online newspapers operate like this. Alex Miller’s discussion of the need for immediacy in real-time data management of live broadcasting is particularly relevant. Real-time responsiveness is possible and advantageous with a sympathetic business culture.

What makes for a corporate culture that can successfully welcome and accommodate the emerging landscape of using data?

  1. An ability to understand a data-driven focus can out-perform many, if not all, previous business solutions
  2. An open-mindedness that supports the research, development and experimentation necessary to make best use of big data
  3. A perspective that supports the idea operating in real-time is an excellent way not only to enhance the customer experience but also monitor for problems and quickly correct them. However difficult it can be to negotiate at all levels, business agility is critical
  4. A marketing optimization framework that allows marketers to use the data effectively to make marketing decisions in real-time. Companies with higher conversion rates almost always have better marketing efficiency ratios (net contribution/marketing expenses). These companies understand it’s hard work to accomplish better marketing efficiency ratios, but it’s considerably more lucrative to do so.
  5. Higher standards of accountability throughout the organization, up to and including the CEO. Does the CEO know which factors of the customer experience impact sales, which projects or departments to favor, what truly needs to be done to optimize the marketing efficiency ratio? In a data-driven business climate, the CEO must know these things
  6. Leadership that promotes higher levels of communication, even collaboration, across all teams
  7. A willingness to grant certain decision making powers to smaller teams
  8. A commitment to employing conversion rate marketing principles
  9. An emphasis on making sure data as well as information and insight derived through analytics are flowing to individual teams across the organization so each can make clear decisions and execute in real- or near-real-time
  10. People and processes that foster a culture of risk-taking and ongoing testing

It took very little time to list (or read) these qualities. It will probably take much more time to internalize them so they become ingrained business practice. If you hope to stay competitive, however, just don’t let it take too long.