Search
Register

Big Data Analytics

Decoding Data Lakes

|

 Those familiar with the Business Intelligence space will know of the role of a Data Warehouse. A data warehouse accumulates data from multiple sources, with the objective of providing analytics that drive business decisions.  In today’s world of big data, we have started hearing a lot about data lakes. Data lakes, like a data warehouse, is a storage repository for vast amount of data.  So, then, is a data lake a different implementation of the data warehouse?  In fact, it is quite different. The term data lakes was coined by James Dixon, former CTO of Pentaho.  According to Dixon, data warehousing led to information silos, which could be overcome by data lakes. The data lake metaphor emerged because ‘lakes’ are a great concept to explain one of the basic principles of big data – the...

The Key to Building Data Pipelines for Machine Learning: Support for Multiple Engines

|

As a consumer of goods and services, you experience the results of machine learning (ML) whenever the institutions you rely on use ML processes to run their operations. You may receive a text message from a bank requiring verification after the bank has paused a credit card transaction. Or, an online travel site may send you an email that offers personalized accommodations for your next personal or business trip. The work that happens behind the scenes to facilitate these experiences can be difficult to fully realize or appreciate. An important portion of that work is done by the data engineering teams that build the data pipelines to help train and deploy those ML models. Once focused on building pipelines to support traditional data warehouses, today’s data engineering teams now build mo...

Unlocking the Power of BIG DATA !

|

 “Data is a precious thing and will last longer than the systems themselves.” –  Tim Berners-Lee, Inventor of the World Wide Web” Big Data is the need of today’s hour. The use of Big Data is becoming common these days by the companies to outperform their peers. We are currently in a data-driven economy where no organization can survive without analyzing the current and future trends. It helps the organizations to combine and analyze the industry data. There are ample of information that companies have about the products, services, buyers and suppliers, consumer preferences which can be put together to analyze. So it is rightly said by Napoleon Bonaparte, “War is 90% information.” Let’s see how much true it is. Big Data can serve to deliver benefits in various domains- Insurance – As big da...

5 Techniques to draw insights from data

|

“Data is the new oil.” — Clive Humby Clive Robert Humby OBE is a British mathematician and entrepreneur in the field of data science and customer-centric business strategies. It is rightly pointed by him about data. Each & every industry/business is standing on the pillars of data. Facebook, Jio – two separate industry players yet united by Data to take on the globe. Data is driving business & making them run faster than ever.  Below are the 5 techniques used by almost all industries to draw insights from Data.  Analytics One of the widely used tools to derive actionable insights from your data, is analytics.  Let us understand what Analytics is. It is the practice of managing, capturing & deriving meaningful insights by turning raw data into information.  For example, it...

DStreams vs. DataFrames: Two Flavors of Spark Streaming

|

This post is a guest publication written by Yaroslav Tkachenko, a Software Architect at Activision. Apache Spark is one of the most popular and powerful large-scale data processing frameworks. It was created as an alternative to Hadoop’s MapReduce framework for batch workloads, but now it also supports SQL, machine learning, and stream processing. Today I want to focus on Spark Streaming and show a few options available for stream processing. Stream data processing is used when dynamic data is generated continuously, and it is often found in big data use cases. In most instances data is processed in near-real time, one record at a time, and the insights derived from the data are also used to provide alerts, render dashboards, and feed machine learning models that can react quickly to new t...

From Data to Decision: The Digital Marketer’s Journey

|

“We must move from numbers keeping score, to numbers that drive better actions.” –David Walmsley, Chief Customer Officer at House of Fraser What is the hype today around data, digital strategy and informed decision making? Why is everyone, including the big names in every industry, gushing towards leveraging the power of data-driven decisions to bring superior and indisputable value to their client’s tables? How can the exorbitant amount of data and information available to us, help drive business with assured returns and productivity? One thing is for sure, metrics do matter, but there is a lot of confusion on which ones. Gathering numbers just to “keep score” is a fruitless tactic today. Decoding the numbers and data through analytics into valuable insights and further extracting sense f...

Demystifying Tech for the TECHADE: Big Data & Analytics (BDA)

|

With plethora of data at our disposal, big data and business analytics solutions are bound to witness an increasing enterprise-wide demand. This demand is expected to be led by strong executive-level initiatives targeted at faster, decision-contextualised, and predictive insights, which could be drawn from the available data. The data could be business data, consumer data, and machine-to-machine data, analysed on cognitive platforms, at decentralised locations, and often for real-time impact. Big data analytics can help businesses make faster data-driven decisions, improve operational efficiency and reduce costs by understanding consumer behaviour. Source: SAS Global State-of-the-Market for BDA As per IDC, the global BDA solutions revenue stood at USD 189 bn. in 2019 and is expected to rea...

The Future of Big Data and Machine Learning Is Clear: It’s All on the Cloud

|

Last week’s announcement that Cloudera and Hortonworks will merge to form a single entity speaks volumes about the state of the big data and machine learning (ML) market: the cloud is the future, and the world of on-premises data centers is becoming a thing of the past. Companies who build their software products for that legacy world have all stopped growing — a notable contrast to the rapid expansion of cloud-native software businesses. On a personal note, this validates what my co-founder Joy and I had predicted when we decided to build Qubole as a cloud-only platform. Many studies including a recent one from Forrester assert big data in the cloud is quickly rising and is expected to grow nearly 7.5 times faster than the on-premises market. The announced merger confirms this analysis an...

Secure Your Big Data in the Cloud with Access Controls

|

Big data has become an essential requirement for enterprises looking to harness their business potential. The use cases for big data are endless and range from customer targeting and fraud analytics to anomaly detection and more. This data can be generated quickly from various sources such as users’ browser and search history, credit card payments, mobile pinging of the nearest cell phone tower, etc. Given the volume of sensitive information being captured, any unauthorized or accidental disclosure of or access to the data can have severe consequences for your enterprise, both in financial terms and in more intangible ways, such as the loss of brand recognition and users’ trust. In recent years, many highly scalable and complex processing frameworks for big data have emerged, such as Hadoo...

The Best Practices of Data Migration Strategies of Businesses

|

Every organization at some point or the other goes through the process of data migration due to various reasons. The basic idea of data migration is a change in the storage space, it can be moving from one server to another, or transferring data from one operating system to another or from one database to another. Whatsoever data migration system the organizations follow, the basic process involves three basic steps, Extract/Transform/Load (ETL), of which the last two steps of Transform and Load are followed during all migration processes. Hence it is clear that the extracted dada goes through several procedures for preparation; only then it can be loaded to the target location. The reasons may vary from an overhaul of the overall system, up-gradation of the existing database, merger of ne...