Big Data: Modeling, Technology and Architecture
Start Date: 16/02/2022
Course Overview
Today companies have the capability to collect large amount of data. Handling large amount of data requires new technologies that are able to collect, cleanse, process and store effectively significant amount of information.
Many companies reached the conclusion that not using this collected data is actually loosing large amount of money. Big Data market is estimated to surpass $200 billion this year.
There is a tremendous business in Big Data and with the right methodologies and tools this row Data can be available for use.
This course provide the basis for Big Data and NoSQL DB environment, architecture, process and available tools. The course will also present Big Data methodologies and deployment recommendations
Who should attend?
Developers, Architects, Product Managers and Managers who whats to know about Big Data.Prerequisite:
None
Course Outline:
1. Introduction
• Definition: Big Data, NoSQL
• The need for Big Data technology
• Tradition technologies Vs Big Data technologies
2. Big Data Architecture
3. Data Collection & Ingestion
• Streaming Concept
– Rest API
• Apache Kafka
– AWS Kinesis, Azure Event Hub
• Apache Flume
• Log Stash
• Commercial solutions – Splunk, Logz.io
4. Hadoop
• What is Hadoop?
• Hadoop Architecture
• Hadoop File System (HDFS)
– Architecture
– NameNode & DataNode
• Hadoop MapReduce
• Apache YARN
• Apache Oozie, Sentry, Tez, HCatalog, ZooKeeper, Ambri, Knox, Falcon
• Hadoop Distribution
– Examples: Cloudera, Hortonworks
• Hadoop Performance Best Practices
5. Apache Pig!
6. Apache Storm
7. Apache Spark
• Concept & Architecture
• Programming with Spark
• Spark Streaming
• Spark SQL, Datasets, and DataFrames
• MLlib
• GraphX
8. Big Data DB types
• Redis10. Column Family Stores (Wide Column Stores)
• Apache HBase
• Apache Cassandra11. Document Databases
• MongoDB
– Architecture & Data Model
– JSON query
– Performance Best Practices12. Graph Databases
• Mathematical Graph as a DB
• Architecture and components
13. ‘SQL’ over Hadoop
• Apache Pig!
• Apache Sqoop
• Apache Hive
– Architecture – Batch Processing
– Apache Impala
– Massively Parallel Processing (MPP)
14. Big Data Deployment
• Local Data Center
• Hosting Services
– AWS, Azure, Google
• Pros and Cons
15. Big Data Northbound Interfaces
• Big Data to OLAP
• BI Visualization
• Scaling BI over Big Data
16. Trends & Conclusions
