What is the concept of big data?

The world contains an unimaginable amount of digitized information, becoming more and more rapidly... From business to science, from government to art, this impact is everywhere. Scientists and computer engineers have coined a new term for this phenomenon: "big data."

So-called big data, what exactly is big data, where does it come from, and what is its definition?

1: The definition of big data.

1. Big data, also known as huge amounts of data, refers to the amount of data involved that is so large that it cannot be captured, managed, and processed by the human brain or even mainstream software tools within a reasonable time. , and organize it into information that helps companies make more positive business decisions.

2. Big data technology refers to the ability to quickly obtain valuable information from various types of big data, including data collection, storage, management, analysis and mining, visualization and other technologies and its integration. Technologies applicable to big data include massively parallel processing (MPP) databases, data mining grids, distributed file systems, distributed databases, cloud computing platforms, the Internet, and scalable storage systems.

The Internet is a magical network, and big data development is also a model. If you really want to understand big data, you can come here. The starting number of this mobile phone is 187, the middle number is 3er0 and the last number It's 14250, you can find it by combining them in order. What I want to say is, unless you want to do it or understand the content, if you are just joining in the fun, don't come.

3. Big data application refers to the act of integrating and applying big data technology to a specific big data collection to obtain valuable information. For different businesses in different fields, different enterprises, or even the same business of different enterprises in the same field, due to differences in their business requirements, data collection and analysis and mining goals, the big data technology and big data used are different. Data information systems can also vary considerably. Only by adhering to the synchronous development of the trinity of "objects, technology, and applications" can the value of big data be fully realized.

When your technology reaches its limit, that is the limit of data. "Big data is not about how to define it, the most important thing is how to use it. The biggest challenge lies in which technologies can better use data and how big data can be used." How is the application of data? Compared with traditional databases and the rise of open source big data analysis tools such as Hadoop, what is the value of these unstructured data services?

Two: Big data. Types and value mining methods

1. The types of big data can be roughly divided into three categories:

1) Traditional enterprise data: including CRM

systems’ consumer data, traditional ERP data, inventory data and accounting data, etc.

2) Machine-generated/sensor data: including call records (CallDetail)

< p>Records), smart meters, industrial equipment sensors, equipment logs (usually Digital exhaust), transaction data, etc.

3) Social data: including user behavior records, feedback data, etc. Social media platforms such as Twitter and Facebook.

2. There are four main methods of mining business value from big data:

1) Segment customer groups, and then provide a solution for each group. Customize special services.

2) Simulate the real environment, explore new needs and improve the return on investment.

3) Strengthen departmental connections and improve the entire management chain and industry. Efficiency of the chain.

4) Reduce service costs and discover hidden clues to innovate products and services.

Three: Characteristics of big data

The industry usually uses it. The four Vs (i.e. Volume, Variety, Value, and Velocity) summarize the characteristics of big data.

Specifically, big data has four basic characteristics:

1. It is a huge data volume

Large data volumes (volumes) refer to large data sets, generally 10TB About the scale, but in practical applications, many enterprise users put multiple data sets together, forming a PB-level data volume;

Baidu data shows that its new homepage navigation needs to provide data every day More than 1.5PB (1PB=1024TB), if printed out, this data would exceed 500 billion sheets of A4 paper. Data confirms that so far, the data volume of all printed materials produced by humans is only 200PB.

2. The data categories are large and diverse.

The data categories (variety) are large, the data comes from a variety of data sources, and the data types and formats are becoming increasingly rich, which has broken through the previous limitations. Structured data category includes semi-structured and unstructured data. Today's data types are not only in the form of text, but also in the form of pictures, videos, audios, geographical location information and other types of data. Personalized data accounts for the absolute majority.

3. The processing speed is fast

Even when the amount of data is very large, it can also process the data in real time. Data processing follows the "1 second law" and can quickly obtain high-value information from various types of data.

4. High value authenticity and low density

High data authenticity (Veracity), with the interest in new data sources such as social data, enterprise content, transaction and application data, etc. , the limitations of traditional data sources have been broken, and enterprises increasingly need effective information power to ensure its authenticity and security. Take video as an example. For a one-hour video, during uninterrupted monitoring, there may be only one or two seconds of useful data.

Four: The role of big data

1. The processing and analysis of big data is becoming the node of the new generation of information technology integration and application

Mobile Internet, things The Internet, social networks, digital homes, e-commerce, etc. are the application forms of the new generation of information technology, and these applications continue to generate big data. Cloud computing provides storage and computing platforms for these massive and diverse big data. Through the management, processing, analysis and optimization of data from different sources, and feeding the results back to the above applications, huge economic and social value will be created.

Big data has the power to bring about social change. But unleashing this energy requires rigorous data governance, insightful data analysis, and an environment that inspires management innovation (Ramayya

Krishnan, Dean of Heinz College, Carnegie Mellon University).

2. Big data is the new engine for the sustained and rapid growth of the information industry

New technologies, new products, new services, and new business formats facing the big data market will continue to emerge. In the field of hardware and integrated equipment, big data will have an important impact on the chip and storage industries, and will also give rise to integrated data storage and processing servers, memory computing and other markets. In the field of software and services, big data will trigger the development of rapid data processing analysis, data mining technology and software products.

3. The use of big data will become a key factor in improving core competitiveness

Decision-making in all walks of life is changing from "business-driven"

To "data-driven" ". The analysis of big data can enable retailers to grasp market dynamics in real time and respond quickly; it can provide decision-making support for merchants to formulate more accurate and effective marketing strategies; it can help companies provide consumers with more timely and effective marketing strategies. Personalized services; in the medical field, it can improve diagnostic accuracy and drug effectiveness; in the public sector, big data has also begun to play an important role in promoting economic development and maintaining social stability.

4. The methods and means of scientific research in the era of big data will undergo major changes

For example, sample survey is a basic research method in social sciences. In the era of big data, the massive behavioral data generated by research objects on the Internet can be mined and analyzed through real-time monitoring and tracking, revealing regularities and proposing research conclusions and countermeasures.

Five: The business value of big data

1. Segmentation of customer groups

"Big data" can segment customer groups and then segment each customer group. Groups take unique actions tailored to their needs. Targeting specific customer groups for marketing and services has always been the pursuit of businesses.

The massive amount of data stored in the cloud and the analysis technology of "big data" make it possible to conduct real-time and extreme segmentation of consumers with extremely cost-effective capabilities.

2. Simulate reality

Use "big data" to simulate reality to discover new needs and improve the return on investment. Sensors are now embedded in more and more products, and the proliferation of cars and smartphones has led to an explosion in the amount of data that can be collected. Social networks such as Blog, Twitter, Facebook and Weibo are also generating massive amounts of data.

Cloud computing and "big data" analysis technology allow merchants to store and analyze these data together with transaction behavior data in real time with high cost efficiency. Transaction processes, product usage and human behavior can all be digitized. "Big data" technology can integrate these data for data mining, and in some cases use model simulation to determine which plan to invest in the case of different variables (such as different promotion plans in different regions)

The highest return.

3. Improve the return on investment

Increase the sharing of "big data" results among relevant departments, and improve the return on investment of the entire management chain and industrial chain. Departments with strong "big data" capabilities can share "big data" results with departments with weaker "big data" capabilities through cloud computing, the Internet and internal search engines, helping them create business value with "big data".

4. Data storage space rental

Enterprises and individuals have massive information storage needs. Only by properly storing data can it be possible to further tap its potential value. Specifically, this business model can be subdivided into two categories: personal file storage and enterprise users. Mainly through easy-to-use APIs, users can conveniently place various data objects in the cloud, and then charge based on usage like water and electricity. At present, many companies have launched corresponding services, such as Amazon, NetEase, Nokia, etc. Operators have also launched corresponding services, such as China Mobile's Caiyun business.

5. Manage customer relationships

The purpose of customer management applications is to deeply analyze and understand customers from different angles based on their attributes (including natural attributes and behavioral attributes), so as to This can increase new customers, improve customer loyalty, reduce customer churn

rates, increase customer consumption, etc. For small and medium-sized customers, specialized CRM is obviously large and expensive. Many small and medium-sized businesses use Fetion as a primary CRM. For example, add old customers to the Fetion group, publish new product previews, special sales notifications, and complete pre-sales and after-sales services in the group's circle of friends.

6. Personalized and accurate recommendations

Within operators, it is common to recommend various services or applications based on user preferences, such as application store software recommendations, IPTV video program recommendations, etc. Through intelligent analysis algorithms such as correlation algorithms, text summary extraction, and sentiment analysis, it can be extended to commercial services and use data mining technology to help customers carry out precision marketing. Future profits can come from customers. Share of the value-added part.

Take the daily "junk text messages" as an example. Not all messages are "junk". They are considered junk because the recipients do not need them. After analyzing the user behavior data, the required information can be sent to the people who need it, so that the "spam text messages" become valuable information. At McDonald's in Japan, users download coupons on their mobile phones and then go to the restaurant to pay with operator DoCoMo's mobile wallet discount. Operators and McDonald's collect relevant consumption information, such as what burgers they often buy, which stores they go to, and how often they consume, and then accurately push coupons to users.

7. Data search

Data search is not a new application. With the advent of the "big data" era, the demand for real-time and full-scale search has also become getting stronger and stronger. We need to be able to search various social networks, user behavior and other data. Its commercial application value is to connect real-time data processing with analysis and advertising, that is, real-time advertising business and social services for in-app mobile advertising.

The user online behavior information mastered by operators makes the data obtained "more comprehensive" and more commercially valuable. Typical applications include China Mobile’s “Pangu Search”.

Six: The important impact of big data on the economy and society

1. It can promote the realization of huge economic benefits

For example, its contribution to the growth of net profits in China’s retail industry, Reduce manufacturing product development and assembly costs, etc. It is estimated that global big data will directly and indirectly drive information technology expenditures to reach US$120 billion in 2013.

2. It can promote and enhance the level of social management

The application of big data in the field of public services can effectively promote the development of relevant work and improve the decision-making level and service efficiency of relevant departments. and social management level, generating huge social value. Many European cities improve urban traffic conditions by analyzing traffic flow data collected in real time to guide drivers to choose the best route.

3. Without high-performance analysis tools, the value of big data will not be released

We must maintain a clear understanding of big data applications and neither be superstitious about its analysis results nor Its important role cannot be denied just because it is not completely accurate.

1) Due to various reasons, the data objects analyzed and processed will inevitably include various erroneous data and useless data. In addition, data analysis, artificial intelligence and other technologies that are the core of big data technology have not yet been fully developed. Mature, so the results of big data analysis and processing completed by computers cannot be required to be completely accurate. For example, Google can predict influenza outbreaks faster than professional organizations by analyzing the search content of hundreds of millions of users. However, due to the interference of useless information on Weibo, this prediction has been inaccurate many times. .

2) What must be clearly positioned is that the focus of the role and value of big data is to guide and inspire the innovative thinking of big data users and assist decision-making. To put it simply, if you want to deal with a problem, usually people can think of one method, but big data can provide ten reference methods. Even if only three of them are feasible, it will triple the number of ideas for solving the problem.

Therefore, objectively understanding and playing the role of big data without exaggerating or minimizing it is the prerequisite for accurately understanding and applying big data.

Seven: Finally, Beijing Kaiyun United will give you a summary

Whether the core value of big data is prediction or not, the model of decision-making based on big data has brought success to many enterprises. Come profit and reputation.

1. Analyzing the value chain of big data, there are three models:

1) Holding big data, but not making good use of it; more typical ones are financial institutions, telecommunications industries, government agencies, etc.

2) No data, but know how to help people with data use it; more typical ones are IT consulting and service companies, such as Accenture, IBM, Oracle, etc.

3) Both data and big data thinking; typical ones are Google, Amazon, Mastercard, etc.

2. The most valuable things in the field of big data in the future are two things:

1) People with big data thinking, such people can transform the potential value of big data For practical benefits;

2) There are no business areas that have been touched by big data. These are oil wells and gold mines that have not yet been tapped, the so-called blue oceans.

Big data is a typical field where information technology and professional technology, information technology industry and various industries are closely integrated. It has strong application demand and broad application prospects. In order to seize the new opportunities brought by this emerging field, it is necessary to continuously track and study big data, continuously improve the awareness and understanding of big data, and adhere to the coordinated progress of technological innovation and application innovation. , accelerate the development and utilization of big data in various economic and social fields, and promote the country, industry, and enterprise's data application needs and application levels to enter a new stage.