Data Glossary Lab

Apache Spark is an open-source, multifunctional parallel processing framework designed for analyzing and modeling Big Data. Unlike traditional processing tools that handle data on a single machine, Spark enables data and computations to be spread over clusters with multiple nodes. It is the industry standard for high-speed data processing because it primarily operates in-memory, allowing it to process massive datasets up to 100 times faster than older disk-based systems like MapReduce. Spark represents the "Heavy Lifting" capability of an organization, providing the infrastructure necessary to execute complex machine learning and real-time analytics at a global scale.

An algorithm is a sequence of repeatable steps, often expressed mathematically, written by a human and executed by a computer, to solve a certain type of data science problem. In machine learning, algorithms take input data and hyperparameters, learn patterns, and produce predictions.

API stands for Application Programming Interface, a software intermediary that ensures a connection between applications or computers, such as embedding Google Maps in a Rideshare application.

Apache Airflow is a platform to programmatically author, schedule, and monitor workflows; it is not simply a task scheduler. While Cron allows you to run scripts at specific times, Airflow allows you to define complex dependencies between them. It transforms isolated scripts and fragile batch jobs into a resilient, code-based ecosystem.

The Activation Function is a mathematical rule that determines the output of a neuron, taking as input the weighted sum of the neuron's inputs. It decides if and how the neuron should influence the network's output.

An accuracy score is an evaluation metric used to estimate a machine learning model’s performance, showing the ratio of the number of correct predictions to the total number of predictions.

An AI Chatbot is intelligent software that uses natural language processing (NLP) and artificial intelligence to converse with humans through text or voice interfaces.

Artificial Intelligence (AI) is the simulation of human intelligence processes by computer systems. These processes include learning from experience (machine learning), understanding natural language, recognizing patterns, solving problems, and making decisions. AI systems can analyze vast amounts of data, identify trends, and perform tasks that traditionally required human intelligence, such as visual perception, speech recognition, and decision-making.