Категории

Databricks Connect Powered by Spark Connect: Develop and Debug Spark From Any Developer Tool

Опубликовано: 29 Сентябрь 2024
на канале: Databricks
1,821
17

Spark developers want to develop and debug their code using their tools of choice and development best practices while ensuring high-production fidelity on the target remote cluster. However, Spark's driver architecture is monolithic, with no built-in capability to directly connect to a remote Spark cluster from languages other than SQL. This makes it hard to enable such interactive developer experiences from a user’s local IDE of choice. Spark Connect’s decoupled client-server architecture introduces remote connectivity to Spark clusters and with that, enables interactive development experience - Spark and its open ecosystem can be leveraged from everywhere.

In this session, we show how we leverage Spark Connect to build a completely redesigned version of Databricks Connect, a first-class IDE-based developer experience that offers interactive debugging from any IDE. We show how developers can easily ensure consistency between their local and remote environments. We walk the audience through real-live examples of how to locally debug code running on Databrick. We also show how Databricks Connect integrates into the Databricks Visual Studio Code extension for an even better developer experience.

Talk by: Martin Grund and Stefania Leone

Connect with us: Website: https://databricks.com
Twitter:   / databricks  
LinkedIn:   / databricks  
Instagram:   / databricksinc  
Facebook:   / databricksinc  

play_arrow
373
9

Text Drop from Signboard VFX Blender Tutorial |

Text Drop from Signboard VFX Blender Tutorial |

play_arrow
46,668
1.8 тыс

PERCHÉ FARE TEATRO | Lezione di recitazione #1

PERCHÉ FARE TEATRO | Lezione di recitazione #1
play_arrow
535
5

Asqar Ashurmatov Dadajon Аскар Ашурматов Дадажон

Asqar Ashurmatov Dadajon Аскар Ашурматов Дадажон
play_arrow
2,909
128

THE OBJECTIFIER - Blender Short Film

THE OBJECTIFIER - Blender Short Film
play_arrow
733
4

How To Change Font Color In WordPress Website | WordPress Change Text Color | Change WordPress Text

How To Change Font Color In WordPress Website | WordPress Change Text Color | Change WordPress Text
play_arrow
291
0

Белая милонга. DJ Оля Кощеева.

Белая милонга. DJ Оля Кощеева.
play_arrow
42,399
639

How to Send Your Passport for Canada Visa Stamping

How to Send Your Passport for Canada Visa Stamping
play_arrow
41,870
232

Дом-2: Ряска подстрекатель

Дом-2: Ряска подстрекатель
Похожие видео
play_arrow
Mosaic AI Gateway - Secure and Govern your AI

Mosaic AI Gateway - Secure and Govern your AI
play_arrow
Data + AI Summit 2023

Data + AI Summit 2023
play_arrow
LLM Module 6: LLMOps | 6.7 Notebook Demo

LLM Module 6: LLMOps | 6.7 Notebook Demo
play_arrow
LLM Module 4: Fine-tuning and Evaluating LLMs | 4.13.1 Notebook Demo Part 1

LLM Module 4: Fine-tuning and Evaluating LLMs | 4.13.1 Notebook Demo Part 1
play_arrow
LLM Module 4: Fine-tuning and Evaluating LLMs | 4.10 Task specific Evaluations

LLM Module 4: Fine-tuning and Evaluating LLMs | 4.10 Task specific Evaluations
play_arrow
LLM Module 4: Fine-tuning and Evaluating LLMs | 4.6 Fine Tuning: LLMs as a Service

LLM Module 4: Fine-tuning and Evaluating LLMs | 4.6 Fine Tuning: LLMs as a Service
play_arrow
LLM Module 4: Fine-tuning and Evaluating LLMs | 4.3 Applying Foundation LLMs

LLM Module 4: Fine-tuning and Evaluating LLMs | 4.3 Applying Foundation LLMs
play_arrow
LLM Module 4: Fine-tuning and Evaluating LLMs | 4.4 Fine Tuning: Few-shot Learning

LLM Module 4: Fine-tuning and Evaluating LLMs | 4.4 Fine Tuning: Few-shot Learning
play_arrow
LLM Module 4: Fine-tuning and Evaluating LLMs | 4.2 Module Overview

LLM Module 4: Fine-tuning and Evaluating LLMs | 4.2 Module Overview
play_arrow
LLM Module 4: Fine-tuning and Evaluating LLMs | 4.1 Introduction

LLM Module 4: Fine-tuning and Evaluating LLMs | 4.1 Introduction
play_arrow
LLM Module 3 - Multi-stage Reasoning | 3.7.1 Notebook Demo Part 1

LLM Module 3 - Multi-stage Reasoning | 3.7.1 Notebook Demo Part 1
play_arrow
LLM Module 3 - Multi-stage Reasoning | 3.3 Prompt Engineering

LLM Module 3 - Multi-stage Reasoning | 3.3 Prompt Engineering
play_arrow
LLM Module 3 - Multi-stage Reasoning | 3.1 Introduction

LLM Module 3 - Multi-stage Reasoning | 3.1 Introduction
play_arrow
LLM Module 3 - Multi-stage Reasoning | 3.2 Module Overview

LLM Module 3 - Multi-stage Reasoning | 3.2 Module Overview
play_arrow
LLM Module 2 - Embeddings, Vector Databases, and Search | 2.8.2 Notebook Demo Part 2

LLM Module 2 - Embeddings, Vector Databases, and Search | 2.8.2 Notebook Demo Part 2
play_arrow
LLM Module 2 - Embeddings, Vector Databases, and Search | 2.8.1 Notebook Demo Part 1

LLM Module 2 - Embeddings, Vector Databases, and Search | 2.8.1 Notebook Demo Part 1
play_arrow
LLM Module 2 - Embeddings, Vector Databases, and Search | 2.6 Best Practices

LLM Module 2 - Embeddings, Vector Databases, and Search | 2.6 Best Practices
play_arrow
LLM Module 2 - Embeddings, Vector Databases, and Search | 2.5 Vector Stores

LLM Module 2 - Embeddings, Vector Databases, and Search | 2.5 Vector Stores
play_arrow
LLM Module 2 - Embeddings, Vector Databases, and Search | 2.2 Module Overview

LLM Module 2 - Embeddings, Vector Databases, and Search | 2.2 Module Overview
play_arrow
LLM Module 2 - Embeddings, Vector Databases, and Search | 2.4 Filtering

LLM Module 2 - Embeddings, Vector Databases, and Search | 2.4 Filtering
play_arrow
LLM Module 2 - Embeddings, Vector Databases, and Search | 2.3 How does Vector Search work

LLM Module 2 - Embeddings, Vector Databases, and Search | 2.3 How does Vector Search work
play_arrow
LLM Module 2 - Embeddings, Vector Databases, and Search | 2.1 Introduction

LLM Module 2 - Embeddings, Vector Databases, and Search | 2.1 Introduction
play_arrow
LLM Module 1 - Applications with LLMs | 1.8 Summary

LLM Module 1 - Applications with LLMs | 1.8 Summary
play_arrow
LLM Module 1 - Applications with LLMs | 1.9 Notebook Demo

LLM Module 1 - Applications with LLMs | 1.9 Notebook Demo