Skip to content

Flame Object Cache enhancement #318

@k82cn

Description

@k82cn

Motivation

Currently, Flame implements cache in a naive way: it's only using HashMap via rest api. It does not work for large data, and it's not efficient.

Function Specification

Build a dedicated object cache via Arrow Flight & Polars.

Solutions

  1. For build-in type, just pickle with RunnerRequest directly
  2. For normal object, put to cache after pickle
  3. For DataSet/DataFrame, put to cache directly

Additional context

N/A

Metadata

Metadata

Assignees

Labels

kind/featureNew feature or requestpriority/p1High priorityrferequest for enhancement

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions