# PySparkDataFrameLoader

> **Class** in `langchain_community`

📖 [View in docs](https://reference.langchain.com/python/langchain-community/document_loaders/pyspark_dataframe/PySparkDataFrameLoader)

Load `PySpark` DataFrames.

## Signature

```python
PySparkDataFrameLoader(
    self,
    spark_session: Optional[SparkSession] = None,
    df: Optional[Any] = None,
    page_content_column: str = 'text',
    fraction_of_memory: float = 0.1,
)
```

## Parameters

| Name | Type | Required | Description |
|------|------|----------|-------------|
| `spark_session` | `Optional[SparkSession]` | No | The SparkSession object. (default: `None`) |
| `df` | `Optional[Any]` | No | The Spark DataFrame object. (default: `None`) |
| `page_content_column` | `str` | No | The name of the column containing the page content. Defaults to "text". (default: `'text'`) |
| `fraction_of_memory` | `float` | No | The fraction of memory to use. Defaults to 0.1. (default: `0.1`) |

## Extends

- `BaseLoader`

## Constructors

```python
__init__(
    self,
    spark_session: Optional[SparkSession] = None,
    df: Optional[Any] = None,
    page_content_column: str = 'text',
    fraction_of_memory: float = 0.1,
)
```

| Name | Type |
|------|------|
| `spark_session` | `Optional[SparkSession]` |
| `df` | `Optional[Any]` |
| `page_content_column` | `str` |
| `fraction_of_memory` | `float` |


## Properties

- `spark`
- `df`
- `page_content_column`
- `fraction_of_memory`
- `rdd_df`
- `column_names`

## Methods

- [`get_num_rows()`](https://reference.langchain.com/python/langchain-community/document_loaders/pyspark_dataframe/PySparkDataFrameLoader/get_num_rows)
- [`lazy_load()`](https://reference.langchain.com/python/langchain-community/document_loaders/pyspark_dataframe/PySparkDataFrameLoader/lazy_load)
- [`load()`](https://reference.langchain.com/python/langchain-community/document_loaders/pyspark_dataframe/PySparkDataFrameLoader/load)

---

[View source on GitHub](https://github.com/langchain-ai/langchain-community/blob/a6a6079511ac8a5c1293337f88096b8641562e77/libs/community/langchain_community/document_loaders/pyspark_dataframe.py#L16)