Skip to content

norun9/JFlinkCustomSerializationSchema

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Caution

The Tuple2SerializationSchema class serializes the second element of the tuple as a String. It cannot be used for other purposes.

  1. Build Project:

    1. Please adjust the versions of flink-core and flink-streaming-java in the pom.xml to match the Flink version used in your PyFlink project.
  2. Add the JAR File to PyFlink Environment:

    1. Place the built JAR file in the directory of the PyFlink project.

    2. Add the JAR file to the PyFlink environment using the add_jars method of the StreamExecutionEnvironment class.

      from pyflink.common import Configuration
      from pyflink.datastream import StreamExecutionEnvironment
      from pyflink.table import StreamTableEnvironment
      
      env = StreamExecutionEnvironment.get_execution_environment()
      env.add_jars("path/to/your/jar_file.jar")
      t_env = StreamTableEnvironment.create(env)
      
  3. Use the Java UDF in PyFlink:

    By retrieving an instance of the Java class, you can utilize its methods and integrate it into your PyFlink job.

    from pyflink.java_gateway import get_gateway
    
    gateway = get_gateway()
    java_udf_instance = gateway.jvm.org.serialization.Tuple2SerializationSchema()
    

About

Custom Java Serialization Schema with PyFlink Jobs

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages