Skip to content

Implement comprehensive multi-format DataFrame Spring integration with Spring Data patterns #1322

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 11 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ dependencies {

// experimental, so not included by default:
// api(projects.dataframeOpenapi)
// api(projects.dataframeSpring)

// kover(projects.core)
// kover(projects.dataframeArrow)
Expand Down Expand Up @@ -162,6 +163,13 @@ val modulesUsingJava11 = with(projects) {
)
}.map { it.path }

val modulesUsingJava17 = with(projects) {
setOf(
dataframeSpring,
examples.ideaExamples.springbootDataframeWeb,
)
}.map { it.path }

allprojects {
if (path in modulesUsingJava11) {
tasks.withType<KotlinCompile> {
Expand All @@ -175,6 +183,19 @@ allprojects {
targetCompatibility = JavaVersion.VERSION_11.toString()
options.release.set(11)
}
}
if (path in modulesUsingJava17) {
tasks.withType<KotlinCompile> {
compilerOptions {
jvmTarget = JvmTarget.JVM_17
freeCompilerArgs.add("-Xjdk-release=17")
}
}
tasks.withType<JavaCompile> {
sourceCompatibility = JavaVersion.VERSION_17.toString()
targetCompatibility = JavaVersion.VERSION_17.toString()
options.release.set(17)
}
} else {
tasks.withType<KotlinCompile> {
compilerOptions {
Expand Down
13 changes: 13 additions & 0 deletions data/spring/customers.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
id,name,country,email
1,Alice Johnson,USA,alice@example.com
2,Bob Smith,Canada,bob@example.ca
3,Charlie Davis,USA,charlie@example.com
4,Diana Evans,UK,diana@example.co.uk
5,Edward Wilson,USA,edward@example.com
6,Fiona Brown,Australia,fiona@example.com.au
7,George Miller,Germany,george@example.de
8,Helen Clark,USA,helen@example.com
9,Ian Thompson,Ireland,ian@example.ie
10,Julia Roberts,USA,julia@example.com
11,Kevin Lee,Canada,kevin@example.ca
12,Linda Perez,Spain,linda@example.es
13 changes: 13 additions & 0 deletions data/spring/sales.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
sale_id,customer_id,product,value,date
1001,1,Laptop,1200.50,2025-01-05
1002,2,Phone,799.99,2025-01-10
1003,3,Tablet,450.00,2025-02-14
1004,4,Headphones,149.99,2025-02-20
1005,5,Monitor,299.49,2025-03-01
1006,6,Keyboard,89.99,2025-03-12
1007,7,Mouse,49.95,2025-03-15
1008,8,Smartwatch,199.00,2025-04-01
1009,9,Camera,650.75,2025-04-12
1010,10,Printer,220.00,2025-04-20
1011,11,Speaker,130.00,2025-05-02
1012,12,Router,99.99,2025-05-10
141 changes: 141 additions & 0 deletions dataframe-spring/INTEGRATION_GUIDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# DataFrame Spring Integration Guide

## Quick Start

### 1. Add Dependency

Add the DataFrame Spring module to your project:

```kotlin
// build.gradle.kts
dependencies {
implementation("org.jetbrains.kotlinx:dataframe-spring:${dataframeVersion}")
}
```

### 2. Enable Component Scanning

```kotlin
@Configuration
@ComponentScan(basePackages = ["org.jetbrains.kotlinx.dataframe.spring"])
class AppConfiguration
```

### 3. Use @DataSource Annotation

```kotlin
@Component
class CustomerService {
@DataSource(csvFile = "customers.csv")
lateinit var customers: DataFrame<CustomerRow>

@DataSource(csvFile = "orders.csv", delimiter = ';')
lateinit var orders: DataFrame<OrderRow>

fun analyzeCustomers() {
println("Total customers: ${customers.rowsCount()}")
// Access data using DataFrame API
}
}
```

### 4. Define Your Data Schema

```kotlin
@DataSchema
interface CustomerRow {
val id: Int
val name: String
val email: String
val registrationDate: String
}
```

## Advanced Configuration

### Manual Bean Registration

If you prefer manual configuration:

```kotlin
@Configuration
class DataFrameConfig {
@Bean
fun dataFramePostProcessor() = DataFramePostProcessor()
}
```

### Custom File Locations

Use Spring's property placeholders:

```kotlin
@DataSource(csvFile = "\${app.data.customers.file}")
lateinit var customers: DataFrame<CustomerRow>
```

### Error Handling

The post-processor provides detailed error messages:

```kotlin
// File not found
RuntimeException: Failed to process @DataSource annotations for bean 'customerService'
Caused by: IllegalArgumentException: CSV file not found: /path/to/customers.csv

// Wrong property type
IllegalArgumentException: Property 'data' is annotated with @DataSource but is not a DataFrame type

// CSV parsing error
RuntimeException: Failed to read CSV file 'customers.csv' for property 'customers'
```

## Best Practices

1. **Use meaningful file paths**: Place CSV files in `src/main/resources/data/`
2. **Define data schemas**: Use `@DataSchema` for type safety
3. **Handle initialization**: Use `lateinit var` for DataFrame properties
4. **Validate data**: Add business logic validation after initialization
5. **Resource management**: CSV files are loaded once during bean initialization

## Troubleshooting

### Common Issues

1. **ClassNotFoundException**: Ensure Spring dependencies are available
2. **FileNotFoundException**: Check CSV file paths are correct
3. **PropertyAccessException**: Ensure DataFrame properties are `lateinit var`
4. **NoSuchBeanDefinitionException**: Enable component scanning or register manually

### Debug Tips

- Enable Spring debug logging: `logging.level.org.springframework=DEBUG`
- Check bean post-processor registration: Look for `DataFramePostProcessor` in logs
- Verify CSV file locations: Use absolute paths for testing

## Integration with Spring Boot

```kotlin
@SpringBootApplication
@ComponentScan(basePackages = ["your.package", "org.jetbrains.kotlinx.dataframe.spring"])
class Application

fun main(args: Array<String>) {
runApplication<Application>(*args)
}
```

## Testing

```kotlin
@SpringBootTest
class DataFrameServiceTest {
@Autowired
private lateinit var customerService: CustomerService

@Test
fun `should load customer data`() {
assertTrue(customerService.customers.rowsCount() > 0)
}
}
```
Loading