Causal Inference Basics

Without explicitly stating otherwise, we consider binary treatment.

1. Potential Outcomes

In the Rubin-Neyman potential outcome (PO) framework, the potential outcome \(Y(t)\) denotes what subject \(i\)’s outcome would be if he were to take treatment \(t\). For example, if the treatment is dichotomous such as getting a heart …

more ...

Lessons Learned with Spark External Table

This post discusses a few lessons I learned while working with external tables in Spark.

What is an External Table?

An external table is created when you define a table from files on disk. It is also called unmanaged table. Spark only manages the metadata about the external table. On …

more ...