Matthew Young's Big Data

One example of how data mining and visualisation could be used to solve a problem is to better understand and analyse large data sets. When big companies and organisations collect data from their users, they often store an unimaginable amount of data which can be hard to keep track of. Through using data mining and visualisation tools respectively, the company can first of all comb through the data thoroughly to determine any patterns or trends or make any predictions in the data, then transform those patterns into easy to understand, eye capturing graphics or diagrams. This allows anyone concerned with the use of the data to fully comprehend how it was used and how they can benefit from it. References : https://weber.itn.liu.se/~aidvi/courses/06/dm/StudentPresentations2007/The%20Role%20of%20Visualization%20in%20Data%20Mining.pdf

One tool available to visualise structured and unstructured data is QlikView, a software program developed by Qlik which allows multiple users at one time to interact with data sets in a clear and easy to understand format. Users can update the visually processed data on the go which is often utilised in large group projects where data is constantly being added and stored to analyse. Another tool available to visualise structured and unstructured data is Tableau. This platform allows users across various different industries to easily create graphics and visual diagrams to simplify and understand data whether it’s organised and processed or remains unstructured. References : https://www.toptal.com/designers/data-visualization/data-visualization-tools#:~:text=The%20best%20data%20visualization%20tools%20include%20Google%20Charts%2C%20Tableau%2C%20Grafana,can%20handle%20large%20data%20sets.

One tool used to mine data is Rapid Miner, a popular software program that provides users with the capability to mine large data sets while offering a platform to carefully analyse and prepare that data. It can be useful for creating models of data and sharing them with other users of the software online to gain feedback and advice on how best to visualise the data collected and processed. Another tool available to use to mine data is Oracle Data Mining. Like most Oracle software, it is often used by large tech companies to better predict patterns in large data sets and visualise them clearly. It also allows for a variety of customisations to be made to user profiles and the way that the data is processed, meaning companies can accurately define their objectives and target customers. References : https://www.analyticsinsight.net/the-top-10-data-mining-tools-of-2018/

If a business or organisation finds itself trailing while competing with other rival businesses, big data analysis is often used to gain an insight into the users or customers of that business in order to better understand their desires and then make more informed and smarter business choices to benefit the company and its users. Users will then tend to gravitate towards services and companies that are best suited to their needs resulting in benefits for both parties.   Big data analysis also allows large businesses and organisations to notice trends or patterns that are hard to identify without processing all the data together and comparing data sets. In various industries these patterns and trends are used to predict or prevent large scale events that could drastically affect the industry, and so big data analysis has proven very useful for certain sectors. Resources :  https://www.simplilearn.com/what-is-big-data-analytics-article#:~:text=Big%20Data%20analytics%20is%20a,fra...

 To limit the negative effects of big data, like data breaches and invasions of privacy, I believe more measures have to be taken in order to ensure users of applications that collect personal data are 100% protected when they allow their data to be utilised. This could include ensuring each account created by the user has two factor authentication, which would lower the risk of hackers accessing personal information when a company’s data is leaked. I also believe the punishment for companies who leak data, even if accidental, should be more severe to force large companies and organisations to be more cautious when dealing with personal user data. References: https://bigdatarachel.wordpress.com/category/17-strategies-for-limiting-the-negative-effects-of-big-data/

An advantage of big data for society is that it has driven a variety of aspects of society to be more efficient and advanced, including transport, health care and national security. Improvements have been made through the analysis of data collected regarding transport routes that are used in large cities to provide more services for busier routes and a better quality of transportation. Health care has significantly improved through data processed by health service officials to better understand and identify health concerns or issues and then treat those issues more appropriately. Lastly, national security has improved through the analysis of data related to crime and terrorism statistics where patterns and trends help prevent attacks in major cities. A disadvantage of big data for society is that it has allowed companies to gain a greater insight into the personal lives of anyone who uses services or applications on the web or other networks, which could lead to a lack (or invasion) of...

One advantage of big data for the individual is that services and facilities the individual uses are now far more tailored to their personal interests and needs. This makes application user experiences more enjoyable and often allows for easier navigation of websites and software. Another advantage of big data for the individual is improvements in the quality of health and fitness statistics. Organisations who provide products to track personal health and fitness data have access to millions of users data allowing them to refine that data in order to best benefit the individual compared to others. A disadvantage of big data for the individual is the risk of data breaches and personal data being leaked. Due to the vast amount of users that large organisations collect data from, the volume of data required to be processed can be immense. This makes data breaches pretty common amongst social media companies like LinkedIn and MySpace, and e-commerce corporations like eBay. One of the worst...

I believe big data will be utilised in various aspects of life in the future, even more so in health, transport and the search for other habitable spaces in our solar system. I believe with the advancements in artificial intelligence and the data collected from trial events occurring at the moment, driverless vehicles could be a very real possibility in the years to come. I believe large companies who provide a service to the public will make the best use out of this type of big data usage and have the opportunity to make life a lot easier for everyone. However, this could lead to a lack of jobs and leaves an unclear vision for the future. As space exploration companies such as NASA and Space-X collect and analyse more and more big data throughout the years, I believe huge steps can be made in preserving life on other planets and creating options in a few hundred years for our society to consider other locations in our solar system to live. I find this a very exciting possibility and I...

One example of a contemporary application of big data used in society is the collection, storage and analysis of data related to the population’s health. As witnessed recently with the spread of the Covid-19 virus, trends and patterns are easily recognised when observing data related to public health and this can help prevent events such as pandemics from occurring in the future. It also allows health care professionals to make more educated decisions when dealing with a person’s health concerns due to the billions of previous health record data collected in the past. Another example of a contemporary application of big data used in society is the big data that governments use to analyse and investigate how they can improve life in large cities or towns in modern countries. The data can be collected in various ways, whether it’s through the use of public transport companies, housing utilities such as energy and water companies, or communication and networking companies. Governments aro...

One example of a contemporary application of big data used in science can be found in the data collected from the Large Hydron Collider near Geneva. The machine is a high energy particle collider and is used to research the answers of certain fundamental questions in the field of physics whilst collecting information related to new particle discoveries and observations. The machine contains detectors with over 150 million censors which take snapshots of the paths that the particles take after collision and all of these images have to be stored and processed, resulting in a lot of big data.   Another example of a contemporary application of big data used in science is the data transmitted between the Mars 2020 Perseverance Rover and NASA data centres here on Earth. The rover takes thousands of pictures per sol (one day on Mars) and then sends that data to NASA scientists who store and analyse it thoroughly to investigate its history and to determine if Mars is suitable for habitabil...

One limitation of big data predictive analytical software is that it requires a reliable set of data of good quality in order to properly process and return high standard analysis. Often projects that require a lot of big data analysis spend a lot of time addressing issues in data of a low quality and it can end up using valuable time required for other aspects of a large-scale project. Data must be of the correct format and relevant to what needs processed and a lot of time can be taken up ensuring that’s the case. Another limitation of big data predictive analytical software is that it may sometimes be unintentionally biased unless clear boundaries are set and a clear hypothesis or outcome is predicted beforehand. As the analytical techniques used are predictive, there is no way for the processing software or application to carefully observe each piece of data in a data set and find any unexpected results or information, which can often lead to an unexpected result after analysis is ...

One example of software developed to collect and store vast amounts of data is Hadoop. The Apache Hadoop software library is used for a variety of purposes. These include allowing multiple machines to connect to one server and a faster rate of processing large data sets. Software of this kind is particularly practical when used for large projects involving a number of teams working with the same data. It is free to download and so it can be easily utilised for large-scale software program development or small group projects.   Another example of software developed to collect big data is Cassandra. Similar to Hadoop, Cassandra is another free, open-source database manager. It allows a number of servers to handle and process a vast amount of data shared between multiple systems. It is easily available and is very popular due to the fact it remains open-source. It was initially developed at Facebook as a tool for their application when it was first released. References :  https:/...

One technical requirement for big data is storage space. With the vast amount of data collected between each industry that makes use of big data, it’s difficult to imagine how many megabytes, gigabytes or terabytes of data and information is required to be stored. This is why large corporations and businesses invest so much in reliable database servers and storage systems in order to contain and handle the data they collect. I found it surprising to know that storing big data wasn’t as expensive as I thought it would be and that there are actually low-cost solutions that organisations can use.   Another technical requirement for big data is up to date analysis software that is capable of sorting structured and unstructured data. With the amount of data some companies collect and plan to process, a decent software program or application is required to distinguish which data is structured (clearly defined sets of data) and which is unstructured (data that is not easily searchable and...

One example of a contemporary application of big data used in business is the collection and processing of data related to trading activity and reports provided by chief executives of   companies around the world by commissions and organisations that help to preserve an equitable stock market and help investors work securely. This data often details the current state of a company and its finances and helps these organisations and commissions notice trends in the business/finance industry. It also helps them prevent stock market crashes and similar events. Another example of a contemporary application of big data used in business is computing cloud services which use data centres around the world to process and store large amounts of data that are then analysed and utilised by multinational companies globally. This helps businesses save storage space for important data and allows easy access to it for any business that requires it.   References :  https://www.tutor2u.net/b...