# WhyLabsCallbackHandler

> **Class** in `langchain_community`

📖 [View in docs](https://reference.langchain.com/python/langchain-community/callbacks/whylabs_callback/WhyLabsCallbackHandler)

Callback Handler for logging to WhyLabs. This callback handler utilizes
`langkit` to extract features from the prompts & responses when interacting with
an LLM. These features can be used to guardrail, evaluate, and observe interactions
over time to detect issues relating to hallucinations, prompt engineering,
or output validation. LangKit is an LLM monitoring toolkit developed by WhyLabs.

Here are some examples of what can be monitored with LangKit:
* Text Quality
  - readability score
  - complexity and grade scores
* Text Relevance
  - Similarity scores between prompt/responses
  - Similarity scores against user-defined themes
  - Topic classification
* Security and Privacy
  - patterns - count of strings matching a user-defined regex pattern group
  - jailbreaks - similarity scores with respect to known jailbreak attempts
  - prompt injection - similarity scores with respect to known prompt attacks
  - refusals - similarity scores with respect to known LLM refusal responses
* Sentiment and Toxicity
  - sentiment analysis
  - toxicity analysis

For more information, see https://docs.whylabs.ai/docs/language-model-monitoring
or check out the LangKit repo here: https://github.com/whylabs/langkit

---
Args:
    api_key (Optional[str]): WhyLabs API key. Optional because the preferred
        way to specify the API key is with environment variable
        WHYLABS_API_KEY.
    org_id (Optional[str]): WhyLabs organization id to write profiles to.
        Optional because the preferred way to specify the organization id is
        with environment variable WHYLABS_DEFAULT_ORG_ID.
    dataset_id (Optional[str]): WhyLabs dataset id to write profiles to.
        Optional because the preferred way to specify the dataset id is
        with environment variable WHYLABS_DEFAULT_DATASET_ID.
    sentiment (bool): Whether to enable sentiment analysis. Defaults to False.
    toxicity (bool): Whether to enable toxicity analysis. Defaults to False.
    themes (bool): Whether to enable theme analysis. Defaults to False.

## Signature

```python
WhyLabsCallbackHandler(
    self,
    logger: Logger,
    handler: Any,
)
```

## Extends

- `BaseCallbackHandler`

## Constructors

```python
__init__(
    self,
    logger: Logger,
    handler: Any,
)
```

| Name | Type |
|------|------|
| `logger` | `Logger` |
| `handler` | `Any` |


## Methods

- [`flush()`](https://reference.langchain.com/python/langchain-community/callbacks/whylabs_callback/WhyLabsCallbackHandler/flush)
- [`close()`](https://reference.langchain.com/python/langchain-community/callbacks/whylabs_callback/WhyLabsCallbackHandler/close)
- [`from_params()`](https://reference.langchain.com/python/langchain-community/callbacks/whylabs_callback/WhyLabsCallbackHandler/from_params)

---

[View source on GitHub](https://github.com/langchain-ai/langchain-community/blob/a6a6079511ac8a5c1293337f88096b8641562e77/libs/community/langchain_community/callbacks/whylabs_callback.py#L42)