Jason Atchley : Legal Tech : Gartner Releases eDiscovery Magic Quadrant Report

Jason Atchley : Legal Tech : Gartner Releases eDiscovery Magic Quadrant Report

jason atchley

Gartner Releases E-Discovery Magic Quadrant Report

Report identifies leaders in the space, as well as prevailing trends behind e-discovery market growth and challenges.

Chris DiMarco, Legaltech News

May 20, 2015    | 5 Comments

Read more: http://www.legaltechnews.com/id=1202727035409/Gartner-Releases-EDiscovery-Magic-Quadrant-Report#ixzz3bXRqA7L2

Gartner Researches’ Magic Quadrants are well known for categorizing innovations and defining the prevailing characteristics of tools and services in a specific marketplace. This year’s E-Discovery Software Magic Quadrant has identified the trends in the space, including predictions on the market size of the e-discovery software as well as an assessment of how the key players in the industry are preforming.
Those in the market for a new tool can use the report to quickly determine what vendors do best, and find the right fit for their practice. Twenty of the most popular e-discovery software suites were ranked for their ability to execute and for the completeness of their vision, dividing the group into four categories: challengers, leaders, visionaries and niche players of the e-discovery space.
Rankings:
  • Leaders: kCura Corp., FTI Consulting Inc.’s FTI Technology, Recommind Inc., ZyLAB, HP, Nuix Pty Ltd. and Exterro Inc.
  • Challengers: Epiq Systems Inc., Kroll Ontrack Inc., AccessData Group Inc. and Symantec Corp.
  • Visionaries: Guidance Software, IBM Corp., Catalyst Repository Systems and Microsoft Corp.
  • Niche Players: CommVault Systems Inc., Driven Inc., Ubic Inc., Xerox Corp. and LexisNexis
While 2014’s rankings were considered stagnant for including the same 20 names from 2013 (with only minor movement between categories, this year’s rankings showed the addition of several names as well as the exit of others including: KPMG, Stroz Friedberg and Integreon.
In addition to rating the 20 top performers within those quadrants, the research analyzed the strengths and weaknesses of the group, providing a baseball card-like quick hit of what they do best and how. Legaltech News will dive deeper into that analysis with profiles and conversations with those vendors in the coming weeks.
Past identifying and categorizing leaders in the industry, Gartner’s research also took a holistic look at the industry. According to the research, the growth of the e-discovery marketplace continued unabated in 2014. Gartner estimated that total revenue for the enterprise e-discovery market was approximately $1.8 billion last year, and had a compound annual growth rate of 12 percent.
The growth is attributed to several trends, including a diversifying array of data streams that must be controlled within the context of discovery, as well as a continued desire to move e-discovery to more flexible, less expensive options.
One factor driving the evaluation and implementation of new e-discovery software is migration to Microsoft’s Office 365. According to the report, “Organizations are in the process of migrating email and documents into Office 365 and need to take a step back on what that means to their established e-discovery process and technology application.”
Another interesting trend identified in the research was the increasing proclivity of vendors to provide SaaS models of delivery. While the report warned that many of these are actually misidentified hosted solutions (which connect to a virtual environment via software installed locally rather than being accessible via Web as true SaaS platforms) it suggested that the method makes sense for those who have a variety of cloud-based information streams to consider in their discovery efforts.
According to the report,” This is a new area for e-discovery practitioners. The legal guidance and requirements on how to treat cloud data (social, website, Web email and Internet-of-thing content) within the e-discovery context is lacking. At the moment, organizations are dealing with the cloud data on an ad hoc basis.”

Read more: http://www.legaltechnews.com/id=1202727035409/Gartner-Releases-EDiscovery-Magic-Quadrant-Report#ixzz3bXRw55Hb

Advertisements

Jason Atchley : Compliance : Top 10 Signs Your Compliance Program is in Trouble

Jason Atchley : Compliance : Top 10 Signs Your Compliance Program is in Trouble

jason atchley

Top 10 Signs Your Compliance Program Is In Trouble

A compliant tribute to David Letterman

Ryan McConnell and Meagan Baker, Corporate Counsel

May 21, 2015    | 0 Comments

Read more: http://www.corpcounsel.com/id=1202727087042/Top-10-Signs-Your-Compliance-Program-Is-In-Trouble#ixzz3bMrC0400

©iStock.com/Noppadol_Anaporn
Last night was David Letterman’s farewell to late-night TV. For over 30 years, he has entertained us with his interviews and antics, showing that the mundane can be funny and made unfamiliar. The guy was a class act, and he will be missed.
As a tribute to Dave, we’ve created a Top 10 list focused on the basic structure of a compliance program—tone at the top, policies, risk assessments, training, communication, monitoring and response. Do your best to imagine Rupert Jee of Hello Deli reading aloud: “Top 10 Signs Your Compliance Program Is In Trouble”

10. The company is using a straw poll to monitor the compliance program.

Monitoring a compliance program is a key ingredient to its success. Unless they know the compliance initiatives are working, compliance professionals cannot gauge whether the program is effective and make potential adjustments. Options for monitoring range from using technology tools to evaluate controls or compliance resources to test how well the program is working.

9. The CCO’s license plate reads “LAWRUP.”

When a compliance professional identifies a compliance failure, the program must respond accordingly. If the failure implicates criminal consequences or a large fine, outside counsel may be appropriate. The compliance team, who has a greater understanding of the business and comes without the increased cost, may handle less-complex issues. There is no one-size-fits-all solution.

8. Company policies are an oral tradition that are categorized only as “Before and after the war.”

If you don’t tell employees what a good job looks like, you cannot expect them to perform. The best policies are clear, concise and contain usable teaching aids. Try using universal imagery. Think of icons that speak to your organization and the risks faced. Give employees the tools they need to succeed and make following the policies simple.

7. The CEO’s favorite episode of “Mad Men” is the one in which Joan secures the Jaguar account to become a partner.

Leadership is the best advocate for compliance. When the CEO speaks, people listen. Smart compliance professionals use business leaders to advocate for their program. Teach the CEO what to focus on—create messages and tools to incorporate compliance into presentations and meetings. When the CEO and CCO work together, they can impact the culture of compliance.

6. The company’s online training consists primarily of YouTube excerpts from “The Wire,” with the CCO talking about how all employees need to “re-up.”

At conferences, we hear that live training is always superior to online training. But what about the employee who has been with the company for 20 years and listened to the training program numerous times? Isn’t it better to provide that employee with the changes to the compliance requirements through online training or some other module that does not detract from his day-to-day job? Does that employee need the same training as someone that is new to the company? As with the compliance program itself, training is not a one-size-fits-all proposition.

5. The CCO insists that any bad news be delivered only via texts to his personal cellphone.

It is easy to overlook communication. We all think we communicate well. Emails and texts may be useful, but implementing formal communication in a compliance program takes some work. Effective communication depends on defining the right channels and a thoughtful escalation process.

4. The business folks sing the tune “Bad Boys” from “Cops” every time anyone from compliance walks into a meeting.

The first job in compliance is to understand the business. Compliance does not work without buy-in from the business. What makes the business operate? What are the pressures from different operations and markets? What keeps the COO up at night? Effective programs have strong coordination between compliance and operations, where the operations team sees compliance as a business enabler—not just a cost center.

3. The CCO frequently invokes scenes from the movie “Jumanji” when discussing the company’s risk assessment results.

A proper risk assessment looks forward and evaluates risks that may impact the compliance program according to subject matter. What could go wrong? When could it happen? What are the potential consequences? How do you rate these things? What factors should you use? A risk assessment is not an internal investigation that provides you with historical information about your program. It’s an exercise in predicting and forecasting.

2. Employees in international markets cannot pick the CCO out of a line-up.

CCO visits to an organization’s international markets have a profound impact. These visits increase compliance visibility and leadership awareness of market activities. Different regions have different issues, and to adequately understand and develop a compliance program that mitigates international risk, the CCO has to mingle with employees and collect information on how the program is working.

1. The U.S. Attorney General refers to your company as “a cartel.”

Public perception of a compliance program is important—and that goes double for your regulators. Speaking at compliance conferences and other events not only provides an opportunity to pick up on the best practices of other organizations, but it also allows a CCO to publicly promote the program. Sometimes perception becomes reality in the mind of regulators.
Ryan McConnell and Meagan Baker are lawyers at McConnell Sovany—a compliance and litigation boutique. McConnell is a former assistant United States attorney who, in addition to writing this column, has taught compliance and criminal procedure at the University of Houston Law Center. Baker’s practice focuses on international compliance issues ranging from risk assessments to developing compliance programs. Send your favorite stupid pet trick to rdm@mcconnellsovany.com.

Read more: http://www.corpcounsel.com/id=1202727087042/Top-10-Signs-Your-Compliance-Program-Is-In-Trouble#ixzz3bMrOAkhu

Jason Atchley : Data Breach Liability: Confidentiality vs. Privacy

Jason Atchley : Data Breach Liability: Confidentiality vs. Privacy

jason atchley

Data Breach Liability: Confidentiality vs. Privacy

Glynna Christian and Nikki Mondschein, Corporate Counsel

Read more: http://www.corpcounsel.com/id=1202727103440/Data-Breach-Liability-Confidentiality-vs-Privacy#ixzz3bHYR6x9I

Credit: alengo/iStockphoto.com
IT service providers, particularly cloud service providers, increasingly are resisting unlimited liability for breaches of privacy and data security obligations in their customer agreements. Instead, they offer unlimited liability for breaches of confidentiality, asserting the customer’s risk of a data breach would be covered as a breach of confidentiality, and arguing that unlimited liability for breaches of data protection obligations is simply double dipping.

A Data Breach Is Not Needed to Create Liability

When an IT service provider takes this position, one of the first questions a customer asks is: Assuming that the service provider has access to data that would be covered by privacy and data security laws, what is the risk if the provider breaches the privacy and data security obligations without an actual data breach
In other words, does there need to be a data breach for the customer to incur liability? Unfortunately, the answer is no.
To fully understand the risk of accepting the IT service provider’s position, a customer should identify:
  • The privacy and data protection requirements the customer must satisfy.
  • The likelihood the IT service provider may cause the customer to fail to comply with those requirements.
  • The potential for damages, fines, penalties or other enforcement activity if the customer fails to comply with those requirements—even absent a data breach.

Privacy and Data Protection Requirements

In terms of the privacy and data protection requirements the customer may need to satisfy, the customer should consider legal and regulatory requirements (including regulatory guidance) and industry standards. For example, if a customer collects or processes credit card information, the customer must comply with the Payment Card Industry Data Security Standards (PCI DSS) as well as Visa’s Cardholder Information Security Program (CISP), MasterCard’s Secure Data Protection program (SDP) and Discover Network’s Information Security and Compliance program (DISC). In addition, Massachusetts 201 CMR 17.00 requires a company that owns or licenses personal information of Massachusetts residents to implement and maintain a comprehensive information security program that contains administrative, technical and physical safeguards.
Even if there is no data breach, failing to comply with these standards may subject the customer to enforcement actions by the relevant regulatory authority and/or significant fines.

‘Flow-Through’ Terms

Once a customer identifies the relevant requirements, the customer should ensure that these requirements are expressly passed through to the IT service provider through well-tailored “flow-through” terms. Not only is the customer at risk for liability if the IT service provider causes it to fail to comply with the requirements; simply failing to flow through the requirements may subject the customer to liability for noncompliance.
This is true even if the service agreement includes a confidentiality clause, which generally requires the receiving party to exercise a duty of care to protect confidential information of the disclosing party in a way that is consistent with the measures the receiving party takes to protect its own confidential information. It is often unclear, however, exactly what measures an IT service provider takes. For example, Massachusetts 201 CMR 17.00 specifically requires companies to oversee its service providers, including requiring its service providers by contract to implement and maintain appropriate security measures.
Legal requirements and industry standards are not the only potential risk. The customer also may have contracts in place with its end-user customers and other third parties that would expose it to unlimited liability for breaches of privacy and data security obligations. If the IT service provider only offers unlimited liability for breaches of confidentiality and the IT service provider’s obligation is to comply with its own duty of care standard and not the customer’s standards, the customer may not be able to look to the IT service provider for full recourse if the IT service provider causes the customer to breach these contractual obligations.

A Data Breach Does Not Always Mean a Breach of Confidentiality

Even if there is a data breach, customers may be at risk that the confidentiality provision does not cover the data subject to the breach. Confidentiality provisions often define “confidential information” in a manner that may not encompass all of the data subject to privacy and data security laws. For example, the definition may include only information that is labeled as confidential or that a “reasonable person” would consider to be confidential. In this case, certain types of data, such as IP addresses or geolocation data, are unlikely to be labeled as confidential when disclosed to the IT service provider and may not be something a “reasonable person” would consider to be confidential.
“Confidential information” often is defined to include end-user customer data but not employee data. The IT service provider’s services, however, may include storing or processing employee data. Particularly for services such as cloud-based HR solutions, this may be as simple as receiving employee names, phone numbers, addresses and emails in order to provide technical support.
If the customer discloses personally identifiable information to the IT service provider that is not covered by the definition of confidential information, then a breach of that data would not be a breach of confidentiality for which the IT service provider would have unlimited liability under the service agreement.

Conclusion

The risk of liability for a breach of privacy and data security obligations without a data breach is only increasing. Audit and enforcement activities have continued to increase, an example being the U.S. Department of Health and Human Services Office for Civil Rights’ focus on HIPAA privacy rule violations—with some resulting in civil penalties in the millions. This risk is likely to continue to grow as regulators and states become even more active in setting data protection requirements and enforcing them, including increasing scrutiny of how companies are flowing down protections to third parties.
Customers will want to minimize their risk in deals with IT service providers by (1) including privacy and data security obligations sufficient to satisfy their privacy and data protection requirements; and (2) insisting on uncapped liability for the IT service provider’s breach of those obligations. If the IT service provider simply refuses to accept such unlimited liability and only offers uncapped liability for breaches of confidentiality, the customer may try to reduce its risk by:
  • Including privacy and data security obligations sufficient to satisfy the customer’s privacy and data protection requirements, even if those obligations are subject to a general limitation on liability.
  • Ensuring damages the customer may incur for breach of privacy and data protection obligations, such as regulatory fines, penalties and the like, are not excluded by a sweeping exclusion of liability for consequential damages, even if they are subject to a general limitation on liability.
  • Seeking a heightened liability cap for breaches of privacy and data security obligations in addition to uncapped liability for breaches of confidentiality
  • Defining “confidential information” to ensure it encompasses all personal data the customer may disclose to the IT service provider.
  • Including the right to terminate for convenience without the payment of any early termination charge.
Glynna Christian is a partner in the corporate department of Kaye Scholer’s New York office. She has over 20 years of experience advising Fortune 100, FTSE 100, and a variety of other public and private companies on complex transactions, including mergers and acquisitions, joint ventures, and other forms of strategic investments and partnerships. She also advises on outsourcing and commercial transactions with an emphasis on technology, media and financial services. Nikki Mondschein is an associate in the corporate department of the firm’s New York office and a member of the IP and technology transactions group. She provides strategic advice to clients on corporate and commercial transactions with an emphasis on the technology, software, media, arts and entertainment sectors. She previously worked as corporate counsel at Apple and Nokia.

Read more: http://www.corpcounsel.com/id=1202727103440/Data-Breach-Liability-Confidentiality-vs-Privacy#ixzz3bHYf5Md1

Jason Atchley : Sales : Quick Tips to Optimize Sales Staff Performance

Jason Atchley : Sales : Quick Tips to Optimize Sales Staff Performance

jason atchley

QUICK TIPS TO OPTIMIZE SALES STAFF PERFORMANCE

April 10, 2015
When overseeing sales performance management, managers often focus on their employees’ quotas. While these quotas suggest what employees are capable of, they do not give a comprehensive picture of workers’ potential.
Rather than simply use quotas to quantify employee performance, employers should implement a performance management approach that seeks to unlock the full potential of all their salespeople. A recent Gallup poll revealed companies maximize only 5 percent of their employees. These workers who are the top performers exhibit three significant characteristics that allow them to work at their best: having an employment history of 10 years or more at the same company, being engaged on the job and working at a position that lets them use their natural abilities.
With these characteristics in mind, employers could follow these tips to optimize employee performance:
Match employees’ positions to skills and experience
When employees know they are working a job that is a great fit for their past history and skills, they are more likely to be engaged with the job. Employers should match their employees to jobs that they feel comfortable doing and excel at the most, according to Gallup.
“Gallup’s research shows that employees are most likely to be engaged – and stay with their companies – when they report that their managers understand them and give them the chance to do what they do best every day,” Gallup said in the report. “Managers can help employees find ways to do more of what they’re good at.”
Train and educate employees for career success
Employees expect more out of their employers, especially when it comes to training and skill development opportunities. People who are good at their jobs frequently want to build on their existing abilities, according to Business 2 Community. They want to learn more sales techniques, technology and software and other innovative tools in the industry.
“Naturally curious, persistent types not only see learning as a way to reach their goals more quickly, but see self-development as a way of life,” Business 2 Community stated. “For them, learning and continual growth do not end at a certain age or stage of life but are the essence of life itself, and therefore never ending.”
Employers should ask employees what skills or sales techniques they want to enhance and provide them with corporate resources that will achieve this to increase employee engagement.

Jason Atchley : Sales : 3 Ways to Motivate a Sales Team

Jason Atchley : Sales : 3 Ways to Motivate a Sales Team

jason atchley

3 WAYS TO MOTIVATE A SALES TEAM

May 4, 2015
Motivating a sales team boosts productivity substantially. Providing goals and encouraging salespeople to meet them by offering various types of incentive compensation for their efforts is an effective way to increase the volume of a business’s total sales.
A business can pump up its team using three different methods to incentive employees and bolster the company:
1. Have a contest 
Sometimes a little healthy competition is exactly what a sales team needs to increase productivity. According to Business News Daily, organizing a contest is a fun way to involve all employees and inspire sales staff to reach a quota.
“We’ve tried running sales contests in the past, using various software and tools,” said Rick Hanson, vice president of worldwide sales and field operations at Hewlett-Packard Enterprise Security, according to Business News Daily. “There was a single goal and the reps who achieved that goal were rewarded, usually with money.”
While a financial incentive for the winner may be one way a business can motivate individuals to participate in the contest, there are other noncash alternatives that may work as well – potentially even better. Some potential prizes an operation may consider offering include:
  • Tickets to a sporting event, concert or local festival
  • Time off from work
  • Dinner on the boss
  • A prize basket
  • The chance to become boss for the day
  • Bragging rights
Performance and compensation management software may be a great way to keep track of progress throughout the duration of a contest. While this may be a good way to motivate salespeople, it may wind up awarding the same top-performing reps and discourage others from participating. Consider creating leagues to assign sales reps to based on past performance.
The Harvard Business Review noted offering prizes that aren’t cash may be especially beneficial if a business holds multiple contests among different groups of salespeople.
2. Create a fun environment 
Making work something employees look forward to can serve as a fantastic motivator. A business can set group goals and offer a prize like a pizza party or ping-pong table for the office. Motivating all employees together is also a great way to unify staff and foster strong relationships among employees.
3. Recognize spouses, partners and kids 
Businesses should also consider reaching out to the family of sales reps. Including them in on the fun can encourage them to motivate the team at home as well. This is a fun way to deepen the relationships a business has with its employees, according to Business News Daily.

Jason Atchley : Data Security : Four Ways IT Teams and CIOs Can Improve the Security Status Quo

Jason Atchley : Data Security : Four Ways IT Teams and CIOs Can Improve the Security Status Quo

jason atchley

4 Ways IT Teams and CIOs Can Improve the Security Status Quo

By Perry Dickau
Few conversations are more stressful for IT pros and CIOs than the ones immediately following a data breach. The unauthorized exposure of customer or employee personally identifiable information (PII) or intellectual property/trade secrets is a worst-case scenario for most companies, so it’s only natural that proactive data protection is a priority. As the recent RSA Conference made clear, it’s no longer enough to enlist reactionary security strategies or focus solely on preventing hacks at the perimeter. Instead, your company must minimize threats by protecting data where it lives.
The only way to avoid that awkward post-breach conversation is to stop it from happening in the first place. Here are four important ways to start improving your data security landscape before a breach occurs.
Protect data at its core.
Data lives and moves within a layered ecosystem – from where it is stored, through networks, servers, applications, and firewalls – as it is managed and consumed throughout its lifecycle.  Does your team prioritize the application or firewall layers at the top of the IT stack when you’re developing security protocols? This strategy has been proven largely ineffective, so it’s time to make changes to the security status quo.
Securing the perimeter is an integral part of data protection, but this method alone is one-dimensional and outdated, and it can leave your business powerless against new threats. Eliminating the possible effects of a breach at the center of the IT stack is more effective, and it’s a more prudent use of time and money. Securing PII and other sensitive data where it’s created means that even if an outsider gains entry to your network, he won’t readily be able to steal information.
Don’t follow hackers’ leads.
It’s important to update security protocols as new threats emerge, but this can’t be the only weapon in your arsenal. Status quo security methods can only stop known threats, and playing catch-up to the evolving security landscape is a losing proposition for IT teams and CIOs. Instead, stay ahead of cybercriminals by creating a holistic approach that uses actionable insights to protect infrastructure on both the inside and outside.
Consider the security of your ecosystem, not just your IT.
The Ponemon Institute reports that 78 percent of data breaches are caused by employees saving information in a vulnerable domain or deleting critical files, while hackers are increasingly adroit at getting into corporate IT systems through other paths. For example, back-door approaches like exploiting unused accounts practically invite hackers to gain entry undetected.
Data-aware technology secures data as it’s created, increasing protection and eliminating threats. If your efforts are devoted to keeping threats out, your core ecosystem won’t be prepared to withstand a breach when one manages to get in. The companies that have suffered high-profile hacks in recent months are a major reminder that change is needed in the security industry, and it’s time to do something different in order to get – and stay – ahead of inbound threats.
Accept that you can’t stop a breach in its tracks.
This isn’t an easy thing for a company to accept. CIOs want to know their teams can identify issues as they occur and bring them to a halt instantly. Unfortunately, this isn’t a reality for anyone. Even detecting a breach or hack while it’s happening has proven difficult. Frequently, the only way to discover these issues is through monitoring event logs or after the subsequent fallout. Adopting a security protocol that improves data visualization can help you prevent data breaches before they take hold, which becomes far more valuable than trying to stop them as they occur.

Jason Atchley : Big Data : Why You Can’t Buy a Data Lake

Jason Atchley : Big Data : Why You Can’t Buy a Data Lake

jason atchley

Why you can’t buy a data lake

Data Lake has become one of the latest buzzwords for data management. It is probably the most misunderstood concept. The reason for that is it could be different for each organization. It depends on how much data exists, what the bottlenecks are and how is it going to be used. The good news is that technology exists today that can enable a wide variety of use cases, a good example is the Hadoop ecosystem.
In pharmaceutical and biotechnology research organizations, there has been a explosion of data long before it hit other industries. From sequencing of the human genomes over multiple years to now sequencing multiple genomes per week, that generated terabytes of raw data, which needed to be managed in a  way so it could be analyzed at least as fast as it was getting produced. Now with Hadoop based solutions, this is somewhat a solved problem, but it was not the case when sequencing technologies started evolving more than 10 years ago.  Then there are experiments that get conducted through various departments in the quest to characterize functions of genes for example. Scientists need a flexible environment to store their results, and record their insights and be able to quickly share those insights with other groups within the organization. The experimental data requires a structured data management treatment, but the insights are largely unstructured text that need to be then analyzed as such, which is where a data lake solution can provide value.There are new technologies being introduced in laboratories all the time, and there is a need to quickly integrate the data from these systems. Traditionally, the way to manage experimental protocols and data has bee to develop large enterprise LIMS systems that take several years to develop and are generally obsolete by the time they are deployed. A data lake in this context could be a re-imagined LIMS system that does not require thousands of hours of programming to integrate new sources of experimental data. One can argue you can still dump the output from experiments in a Hadoop environment, but managing samples through a laboratory workflow and seamlessly integrating instrumentation within that workflow  all the while tracking the details of each process is needed for regulatory reasons.  Data lake in a biotechnology research organization has the potential to accelerate productivity by removing bottlenecks in data movement through the organization, but it has to be designed for efficiency like any IT system. A high performance data storage and analysis environment is needed for both exploratory analytical experiments as well as production data analysis processes, both of which can potentially utilize variants of the data lake concept utilizing the Hadoop ecosystem.
In the magazine publishing industry or any consumer retail industry, the understanding the value of efficiently managing and utilizing large amounts of data is in its infancy by comparison to the the biological sciences mentioned above. In most cases, marketers are the key users of data for in any consumer retail industry. In the magazine publishing industry, traditionally, they would sales data and use segmentation to group their customers and maybe use some demographic data from other sources to further refine groups, and then either cross-sell or up-sell or flat out ignore bad payers for example. Very little statistical modeling was involved in this relatively simple model. In the last decade, there has been a explosion of data now available that can potentially characterize these customers even better beyond just the purchase history. The challenge here is to develop a analytical methodology that can extract the signal from the noise. A bigger challenge is even knowing what the signal is you are looking for, and how do you know it is significant. In such a environment, the role of the data lake is really less of data management, more of a playground that enables data scientists to use large data sets and run exploratory analyses to run data experiments, develop new methods and test them rapidly with new sources data as they become available. In this context, the data lake maybe more of a data repository that ingests data from all sources, internal and external, structured and unstructured and provides enough resources for data scientists. Most consumer retail organizations are not as mature as Google, Amazon etc, hence the data lake maybe more of a simplistic solution to get started. The key for such companies is to start small and carefully define specific use cases to be implemented first as they launch the data lake journey.
The current marketing materials from Hadoop vendors will try to convince you they have the data lake strategy for you, but in fact every IT organization in conjunction with the business needs to define what it is going to be. Managing and analyzing large amounts of data is a key requirement for business success now, but it each business needs to define what that means for them. Technology continues to evolve at a rapid pace, utilizing the power of opensource. As new use cases for data and analytics are explored, new components are being added to the Hadoop ecosystem as we speak. There is a lot of confusion and opinions regarding the Hadoop environment due to that. An experience that an organization may have had with Hadoop 2 years ago maybe no longer relevant as the solution set has evolved. Picking a vendor that stays somewhat current with the technology trends is an important consideration, as you launch your data lake strategy.
CIOs and C-level execs are enamored by the data lake term, but unfortunately, it is not something you can just buy off the shelf and check a box: “we now have a data lake”!
%d bloggers like this: