Problem with merging coach data

[ue71603]

In master:

./scripts/scripts_fernbusse.txt run.log all

in the merge_ferbusse block there is a problem.

(1) without the epip_db_to_db.py step there was an error
(2) with the epip_db_to_db.py step there is also an error
2025-02-27 16:57:03,807 - script_runner - INFO - merge_fernbusse - step: 4: epip_db_to_db.py C:/Users/ue71603/Downloads/merge_fernbusse/03-gtfs.duckdb C:/Users/ue71603/Downloads/merge_fernbusse/04-epip.duckdb

If you believe we can do short cuts and omit some steps then I am happy.

2025-02-27 16:57:46,497 - script_runner - INFO - Indexing RoutePoint to ScheduledStopPoint 344.87109375
2025-02-27 16:57:47,572 - script_runner - INFO - Service journeys for now 346.3515625
 ServiceJourney 02025-02-27 16:57:48,394 - script_runner - ERROR - 'NoneType' object has no attribute 'id' Traceback (most recent call last):
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\tool_script_runner.py", line 332, in <module>
    main(args.script_file, args.log_file, args.log_level, args.blockname, args.begin_step)
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\tool_script_runner.py", line 295, in main
    result=load_and_run(script_name, script_args)
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\tool_script_runner.py", line 79, in load_and_run
    result = mod(*args1)
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\epip_db_to_db.py", line 71, in main
    epip_service_journey_generator(source_db, target_db, generator_defaults, None, cache=False)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\transformers\epip.py", line 596, in epip_service_journey_generator
    write_generator(db_write, ServiceJourney, query(db_read), True)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\netexio\dbaccess.py", line 704, in write_generator
    for a in _prepare3(generator, objectname):
             ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\netexio\dbaccess.py", line 676, in _prepare3
    for obj in generator3:
               ^^^^^^^^^^
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\transformers\epip.py", line 579, in query
    yield process(sj, db_read, db_write, generator_defaults)
          ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\transformers\epip.py", line 537, in process
    if service_journey_pattern.id not in sjp_ids:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'id'

[ue71603]

problem also persists with latest version:
./scripts/scripts_fernbusse.txt fb.run all

2025-03-10 11:50:06,053 - script_runner - INFO - merge_fernbusse - step: 4: epip_db_to_db.py C:/Users/ue71603/Downloads/merge_fernbusse/03-gtfs.lmdb C:/Users/ue71603/Downloads/merge_fernbusse/04-epip.lmdb
2025-03-10 11:50:07,189 - script_runner - INFO - Copy all tables as-is 477.17578125
Database Direction does not exist, creating it.
Database DataSource does not exist, creating it.
Database Operator does not exist, creating it.
Database ScheduledStopPoint does not exist, creating it.
2025-03-10 11:50:27,020 - script_runner - INFO - Copy lines, in EPIP style 606.0859375
epip_line_memory
Database Line does not exist, creating it.
2025-03-10 11:50:29,657 - script_runner - INFO - Fix Quay / StopPlace locations 632.29296875
2025-03-10 11:50:31,880 - script_runner - INFO - Investigate this site frame step 648.0625
epip_site_frame_memory
Database PassengerStopAssignment does not exist, creating it.
Database StopPlace does not exist, creating it.
2025-03-10 11:50:35,558 - script_runner - INFO - Service journeys 651.6328125
epip_service_journey_generator
2025-03-10 11:50:36,564 - script_runner - INFO - Indexing RoutePoint to ScheduledStopPoint 661.64453125
2025-03-10 11:50:37,651 - script_runner - INFO - Service journeys for now 661.87890625
Database ServiceJourney does not exist, creating it.
Database ServiceJourneyPattern does not exist, creating it.
Exception ignored in: <generator object load_generator at 0x00000275B32E7370>
Traceback (most recent call last):
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\netexio\dbaccess.py", line 377, in load_generator
    with db.env.begin(write=False, buffers=True, db=db.open_db(clazz)) as txn:
lmdb.Error: Attempt to operate on closed/deleted/dropped object.
2025-03-10 11:50:37,705 - script_runner - ERROR - 'NoneType' object has no attribute 'id' Traceback (most recent call last):
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\tool_script_runner.py", line 356, in <module>
    main(args.script_file, args.log_file, args.log_level, args.blockname, begin_step=args.begin_step,end_step=args.end_step, this_step=args.this_step)
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\tool_script_runner.py", line 317, in main
    result=load_and_run(script_name, script_args)
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\tool_script_runner.py", line 84, in load_and_run
    result = mod(*args1)
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\epip_db_to_db.py", line 71, in main
    epip_service_journey_generator(source_db, target_db, generator_defaults, None, cache=False)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\transformers\epip.py", line 594, in epip_service_journey_generator
    db_write.insert_objects_on_queue(ServiceJourney, query(db_read), True)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\netexio\database.py", line 296, in insert_objects_on_queue
    for obj in objects:
               ^^^^^^^
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\transformers\epip.py", line 577, in query
    yield process(sj, db_read, db_write, generator_defaults)
          ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ue71603\MG_Daten\github\reference1\gtfs-netex-test\transformers\epip.py", line 534, in process
    if service_journey_pattern.id not in sjp_ids:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'id'


Process finished with exit code 0

[ue71603]

the relevant sj: 2025-03-10 12:06:42,908 - script_runner - ERROR - No service journey pattern for journey: ServiceJourney(name_of_class_attribute=None, id='BB:ServiceJourney:1002323173', validity_conditions_or_valid_between=[], alternative_texts=None, data_source_ref_attribute=None, created=None, changed=None, modification=<ModificationEnumeration.NEW: 'new'>, version='20250310', status_attribute=<StatusEnumeration.ACTIVE: 'active'>, derived_from_version_ref_attribute=None, compatible_with_version_frame_version_ref=None, derived_from_object_ref=None, key_list=None, private_codes=PrivateCodes(private_code=[PrivateCode(value='1002323173', type_value='trip_id')]), extensions=None, branding_ref=None, responsibility_set_ref_attribute=None, name=None, short_name=MultilingualString(value='50019', lang=None, text_id_type=None), description=None, distance=None, private_code=None, projections=None, info_links=None, sections_in_sequence=None, transport_mode=None, transport_submode=None, external_vehicle_journey_ref=None, type_of_product_category_ref=None, type_of_service_ref=None, link_sequence_projection_ref_or_link_sequence_projection=None, monitored=None, accessibility_assessment=AccessibilityAssessment(name_of_class_attribute=None, id='BB:AccessibilityAssessment:1002323173', validity_conditions_or_valid_between=[], alternative_texts=None, data_source_ref_attribute=None, created=None, changed=None, modification=<ModificationEnumeration.NEW: 'new'>, version='20250310', status_attribute=<StatusEnumeration.ACTIVE: 'active'>, derived_from_version_ref_attribute=None, compatible_with_version_frame_version_ref=None, derived_from_object_ref=None, extensions=None, mobility_impaired_access=<LimitationStatusEnumeration.TRUE: 'true'>, limitations=None, suitabilities=None, comment=None), journey_accountings=None, notice_assignments=None, occupancies=None, service_alteration=None, departure_time=None, departure_day_offset=None, frequency=None, journey_duration=None, day_types=DayTypeRefsRelStructure(id=None, modification_set=<ModificationSetEnumeration.ALL: 'all'>, day_type_ref=[DayTypeRef(value='', name_of_ref_class=None, created=None, changed=None, version='20250310', modification=None, ref='BB:DayType:1002323173', version_ref=None, uri=None)]), route_ref=None, journey_pattern_ref=ServiceJourneyPatternRef(value='', name_of_ref_class=None, created=None, changed=None, version='20250310', modification=None, ref='OPENOV:ServiceJourneyPattern:352a19b9', version_ref=None, uri=None), time_demand_type_ref=None, timing_algorithm_type_ref=None, journey_frequency_group_ref=None, vehicle_type_ref_or_train_ref=None, vehicle_ref=None, operational_context_ref=None, block_ref=None, course_of_journeys_ref=None, public_code=None, operator_ref_or_operator_view=None, flexible_line_ref_or_line_ref_or_line_view_or_flexible_line_view=LineRef(value='', name_of_ref_class=None, created=None, changed=None, version='20250310', modification=None, ref='BB:Line:2946291122', version_ref=None, uri=None), direction_type=None, journey_pattern_view=None, groups_of_services=None, time_demand_types=None, train_numbers=None, origin=None, destination=None, print=None, dynamic=None, wait_times=None, run_times=None, layovers=None, passing_times=TimetabledPassingTimesRelStructure(id=None, timetabled_passing_time=[TimetabledPassingTime(name_of_class_attribute=None, id='BB:TimetabledPassingTime:1002323173_1', validity_conditions_or_valid_between=[], alternative_texts=None, data_source_ref_attribute=None, created=None, changed=None, modification=<ModificationEnumeration.NEW: 'new'>, version='20250310', status_attribute=<StatusEnumeration.ACTIVE: 'active'>, derived_from_version_ref_attribute='20250310', compatible_with_version_frame_version_ref=None, derived_from_object_ref='BB:Call:1002323173_1', extensions=None, journey_ref_or_special_service_ref_or_service_journey_ref_or_vehicle_journey_ref=None, alight_and_reboard=None, point_in_journey_pattern_ref=StopPointInJourneyPatternRef(value='', name_of_ref_class=None, created=None, changed=None, version='20250310', modification=None, ref='OPENOV:StopPointInJourneyPattern:352a19b9-1', version_ref=None, uri=None, order=1), arrival_time=XmlTime(23, 50, 0, 0), arrival_day_offset=1, departure_time=XmlTime(23, 50, 0, 0), departure_day_offset=1, waiting_time=None, headway=None, latest_arrival_time=None, latest_arrival_day_offset=None, earliest_departure_time=None, earliest_departure_day_offset=None, occupancies=None), TimetabledPassingTime(name_of_class_attribute=None, id='BB:TimetabledPassingTime:1002323173_2', validity_conditions_or_valid_between=[], alternative_texts=None, data_source_ref_attribute=None, created=None, changed=None, modification=<ModificationEnumeration.NEW: 'new'>, version='20250310', status_attribute=<StatusEnumeration.ACTIVE: 'active'>, derived_from_version_ref_attribute='20250310', compatible_with_version_frame_version_ref=None, derived_from_object_ref='BB:Call:1002323173_2', extensions=None, journey_ref_or_special_service_ref_or_service_journey_ref_or_vehicle_journey_ref=None, alight_and_reboard=None, point_in_journey_pattern_ref=StopPointInJourneyPatternRef(value='', name_of_ref_class=None, created=None, changed=None, version='20250310', modification=None, ref='OPENOV:StopPointInJourneyPattern:352a19b9-2', version_ref=None, uri=None, order=2), arrival_time=XmlTime(5, 15, 0, 0), arrival_day_offset=2, departure_time=XmlTime(5, 15, 0, 0), departure_day_offset=2, waiting_time=None, headway=None, latest_arrival_time=None, latest_arrival_day_offset=None, earliest_departure_time=None, earliest_departure_day_offset=None, occupancies=None)]), passenger_at_stop_times=None, parts=None, calls=None, facilities=None, check_constraints=None, passenger_carrying_requirement_ref_or_passenger_carrying_requirements_view=None, train_size=None, equipments=None, flexible_service_properties_ref_or_flexible_service_properties=None)