دوره آموزش Advanced Big Data Analytics

سرفصل و محتوای دوره ی مباحث پیشرفته در تحلیل کلان داده با معماری مدرن

Advance big data analytics with modern architecture - from batch to streaming

معرفی و هدف دوره :

دوره آموزشی مباحث پیشرفته در تحلیل داده های کلان که با تمرکز بر پردازش جریان داده طراحی شده، ابزار ها و تکنیک های پردازش جریان داده بررسی خواهند شد.

هدف این دوره آموزشی بررسی مباحث پیشرفته در حوزه دیتا و big data جهت احراز نیازمندی های مشاغل

-          Data Engineer

-           Data Scientist

-           Analytics Engineer

 می باشد.

Course content at a glance:

Kafka and streaming advances – 6 hours

Java SE basics for Flink readiness – 12 hours

Real-time stream processing with Apache Flink – 15 hours

Data transformation with data build tool(dbt) – 3 hours

      Machine learning and advance analytics with Spark ML – 6 hours

Data governance and quality in practice – 3 hours

مشاهده رزومه مدرس

طول دوره : 45 ساعت

پیش نیاز دوره : 

-          دوره آموزشی Applied Big Data Fundamentals یا دو سال سابقه کاری در زمینه بیگ دیتا

-          آشنایی با یک زبان برنامه نویسی

Course content, details:

1.    Kafka and streaming advances – 6 hours

Real-time data pipeline development with kafka connect

Stream processing with ksqlDB

ksqlDB features, limitations and best practices

Transactional producers

Kafka tired storage

Dead letter queues and exception handling

2.    Java SE basics for Flink readiness– 12 hours

Java history and why Java for streaming

Introduction to Java & Setup

Development environment setup

Building the project

Maven overview

Core Syntax & Data Types

Control Flow & Methods

Classes & Objects

Inheritance & Polymorphism

Encapsulation & Packages

Exception Handling

Debugging & Logging

Java Collections Framework

Generics

Java I/O Basics

Multithreading & Concurrency

Lambdas, functional interfaces

Streams API

Networking Basics

Serialization

Annotations & Reflection

3.    Real-time stream processing with Apache Flink – 15 hours

Stream processing with industry gold standards

Apache Flink overview

DataStream API

Table and SQL API

Flink SQL

Data sources and sinks

Flink kafka source and sink

Flink Clickhouse sink

Flink MongoDB sink

Flink ScyllaDB sink

Flink Iceberg sink

Stateless operations

Stateful operations

Checkpointing and exactly-once delivery

Handling of time

State recovery

4.    Data transformation with data build tool(dbt) – 3 hours

dbt essentials and features

dbt resources and project structure

Building transformation DAGs

dbt Spark adapter

dbt Trino adapter

dbt Flink adapter

5.    Machine learning and advance analytics with Spark ML – 6 hours

Machine learning and ML in distributed manner

ML algorithms in Spark ML

Spark ML pipelines

Case studies and examples for clustering, classification and recommendation

Deep learning with Spark

RAG and LLM essentials

Vector DB

Spark NLP and LLM

Model deployment

6.    Data governance and quality in practice – 3 hours

Data quality management essentials

Data completeness, accuracy, consistency, validity, uniqueness in practice

Why data governance matters

DAMA DMBOK and data governance knowledge areas

GDPR and CCPA in practice