feat/database sync dev #52
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Owner
williscool
commented
Mar 4, 2025
williscool
commented
- feat: WIP database sync
- feat: wip at least android builds!
- fix: build works! sqlite access seems to but not entirely sure yet
should sync the new database to the supabase db I setup but that doesn't work yet will have to find a way to sync the orignal events db with it
can sync with powersync events from the powersync server now gota figure out how to extricate the events from the existing db to the powersync one
also logger for debugging
well it would work if sqlite supported adding primary keys in ALTER TABLE statements... but alas it does not
Owner
Author
|
Willing to bet the issues adding new test events with the dev plug-in thing is the ID being hard-coded and breaking the primary key constraint |
Owner
Author
|
Follow-Up comment from the power sync team https://discord.com/channels/1138230179878154300/1346321577544912936/1348797082030313512 |
Owner
Author
|
Though I do wonder if cr-sqlite is Overkill here and if I could just do a select into with insert or ignore on conflict |
Owner
Author
|
The thing is I want to keep the sync realtime. Gotta figure that out So far I got. Triggers on eventsv9 a mirror cr-sqlite table so I don't have to change schema of current table. Gotta figure out how to observe those changes in realtime to tell power sync to get the latest tables |
Owner
Author
|
Owner
Author
|
The other error right now may be not having a place that can do the websocket connection that it wants to |
Owner
Author
it installs crsqlite... but the app crashes on load now becasue standard android sqlite doesn't support loading extensions o.o so im gonna try replacing standard sqlite to see if that fixes it
if we were loading crsqlite from the right place I think this round might work
but all the stuff seems like the db is really jacked up definitely backup your db and don't push this to a real device yet still the wild west cowboy stuff until I say differnt
Owner
Author
|
Latest thing I did had to manually upload the cr-sqlite.so into the data dir Not sure why it couldn't find it in the native modules dir. But whatever |
now to fix the connect issues when that requery seems to have when we are not on the react native view
instead of having to upload to data dir. also able to get rid of the unnecssary loading code in events storage now that we do it at connection level
On each connection!
classCustomUse may not be needed but easier just to keep the api the same everywhere. to be clear I think because we use customUse on all SQLiteOpenHelper internal SQLiteDatabase db call we could probably juse use .use on the SQLiteOpenHelper subclasses but easier to just say customUse everywhere because you MUST use customUse for all SQLiteDatabase calls
Owner
Author
|
Okay it works without as many crazy errors now. Now for the main event getting the server sync working. I want to add a user interface where you can press a button to do the database migration for CR SQL lite or not. Where it says use it your own risk. Please back up your database first LOL I also want a UI where you have to set the journey app link and your development token. And once you submit that it will give you a debug view that will show the stuff setting up and things like that. I don't know if it can run in the background while the react app is not running, but we can figure that out later |
Owner
Author
|
https://docs.expo.dev/versions/latest/sdk/background-fetch/ Is probably helpful for that |
the postgres schema to support numbers large enough
…ency issues This commit addresses the dependency error caused by removing react-native-safe-area-context while still using react-native-screens with React Navigation. - Created a mock implementation of react-native-safe-area-context to satisfy React Navigation's import requirements without the native code that was causing build issues - Split the mock implementation into template files for easier review and maintenance: * module-index.js - ES modules implementation * commonjs-index.js - CommonJS implementation * typescript-index.d.ts - TypeScript definition * package.json - Package metadata - Added setup script (setup_safe_area_mock.js) that generates the mock module from templates - Added postinstall hook to package.json to automatically setup the mock after dependencies are installed - Updated CI workflow to ensure the mock is properly set up before building - Updated README with documentation about the mock implementation, its structure, and purpose This is a temporary workaround until we can update SDK versions to properly support both packages without breaking the build.
williscool
force-pushed
the
feat/database_sync_dev
branch
from
148dcbd to
0827fb6
Compare
- Add ScrollView to ensure all content is accessible - Implement toggleable debug section to keep UI elements visible - Default debug panel to collapsed state for better UX
|
Build artifacts for this PR are available: You can download these artifacts from the "Artifacts" section of the workflow run. |
1 similar comment
|
Build artifacts for this PR are available: You can download these artifacts from the "Artifacts" section of the workflow run. |
- Add tri-state connection status logic (null|true|false) to handle initializing state - Show informative blue banner during PowerSync initialization - Only disable sync buttons when definitely disconnected, not during initialization - Use PowerSync API's hasSynced property to properly detect initialization complete - Add visual indicators to distinguish between initializing and disconnected states - Prevent false disconnection warnings during PowerSync startup
Owner
Author
|
done:
|
|
Build artifacts for this PR are available: You can download these artifacts from the "Artifacts" section of the workflow run. |
1 similar comment
|
Build artifacts for this PR are available: You can download these artifacts from the "Artifacts" section of the workflow run. |
|
Build artifacts for this PR are available: You can download these artifacts from the "Artifacts" section of the workflow run. |
1 similar comment
|
Build artifacts for this PR are available: You can download these artifacts from the "Artifacts" section of the workflow run. |
Added support for arm64-v8a architecture required for Pixel 9 Pro and other modern Android devices. This includes: 1. **Native Library Support** - Created script to build crsqlite.so for arm64-v8a architecture - Added proper directory structure for ARM64 native libraries 2. **Build Configuration** - Updated Android build.gradle to properly configure ABI filters - Added APK splitting for optimized app size by architecture - Fixed syntax in abiFilters declaration 3. **Documentation** - Added detailed instructions for building native libraries for ARM64 devices - Enhanced troubleshooting section with common issues and solutions - Added specific instructions for Pixel 9 Pro support 4. **Toolchain Management** - Added safeguards to ensure nightly Rust toolchain is used - Documented issues with Homebrew Rust vs. official Rust installations - Added validation for submodule initialization - Full arm64-v8a support for native libraries - Optimized APK size with architecture-specific builds - Streamlined build process with new convenience scripts: - `yarn build:crsqlite:arm64` The changes ensure proper support for modern devices like Pixel 9 Pro while maintaining compatibility with x86_64 emulators and adding robust error handling and documentation for future developers.
- Add support for split APKs by architecture (arm64-v8a, x86_64) - Process both debug and release APKs for different architectures - Implement clear naming convention for architecture-specific builds - Update artifact uploads to include all architecture variants - Enhance logging for better debugging and troubleshooting - Update PR comments to indicate available architecture-specific builds - Ensure GitHub releases include all architecture variants
|
Build artifacts for this PR are available: You can download these artifacts from the "Artifacts" section of the workflow run. |
1 similar comment
|
Build artifacts for this PR are available: You can download these artifacts from the "Artifacts" section of the workflow run. |
|
Build artifacts for this PR are available: You can download these artifacts from the "Artifacts" section of the workflow run. |
1 similar comment
|
Build artifacts for this PR are available: You can download these artifacts from the "Artifacts" section of the workflow run. |
williscool
commented
Mar 21, 2025
Owner
Author
williscool
left a comment
williscool
left a comment
There was a problem hiding this comment.
Mane that was hard as hellllll!! Project pat gif.jpg
But it works!!!!
williscool
added a commit
that referenced
this pull request
Mar 21, 2025
* feat: WIP database sync
* feat: wip at least android builds!
* fix: build works! sqlite access seems to but not entirely sure yet
* it compiles and runs but still working on it.
should sync the new database to the supabase db I setup but that doesn't
work yet
will have to find a way to sync the orignal events db with it
* it WORKS!@!@!@!@!@!@!@@@!!@o.o !@!@!!!!!
can sync with powersync events from the powersync server
now gota figure out how to extricate the events from the existing db to
the powersync one
* setup context and useQuery for live updates
also logger for debugging
* IT ALMOST WORKS!@!@!@!!!!!!!!
well it would work if sqlite supported adding primary keys in ALTER
TABLE
statements... but alas it does not
* the beginning of some cowboy shit I might rollback :)
it installs crsqlite... but the app crashes on load now becasue standard
android sqlite doesn't support loading extensions o.o
so im gonna try replacing standard sqlite to see if that fixes it
* the LATEST ROUND OF IT WOULD work... but doesn't :/
if we were loading crsqlite from the right place I think this round
might work
* IT uh.. AT LEAST LOADS THE APPLICATION!!!!!
but all the stuff seems like the db is really jacked up definitely
backup your db and don't push this to a real device yet
still the wild west cowboy stuff until I say differnt
* ok we have cr-sqlite installed on the table and the ap doesnt crash!
now to fix the connect issues when that requery seems to have when we
are not on the react native view
* fix: now we load the prebuilt crsqlite properly
instead of having to upload to data dir.
also able to get rid of the unnecssary loading code in events storage
now that we do it at connection level
* fix: a function that might actually close crsqlite properly
On each connection!
* fix: customUse for open helper
* finally works without the unfinished transaction errors
classCustomUse may not be needed but easier just to keep the api the
same everywhere.
to be clear I think because we use customUse on all SQLiteOpenHelper
internal SQLiteDatabase db call we could probably juse use .use on the
SQLiteOpenHelper subclasses but easier to just say customUse everywhere
because you
MUST use customUse for all SQLiteDatabase calls
* fix: setting up table for crsqlite works!
* some upload stuff that would work... if I had dataUpload working
* fix: implemented Subapase connector! uploading dummy event works!
* fix: would extract from sqlite to postgres! ..but I need to update
the postgres schema to support numbers large enough
* ci: maybe fix build
* chore: refactor cr-sqlite stuff into its own lib
also disable it for now because we are only going to do unidirectionaly
sync at first
then we come back around for bi directional which it will be used for
* chore: major refactor
- Split index.tsx into separate components for better organization
- Rename HelloWorld to SetupSync to better reflect its purpose
- Move App component to App/index.tsx
- Move SetupSync component to App/SetupSync.tsx
- Wrap async operations in IIFEs for proper error handling
- Clean up imports and remove unused ones
- Maintain existing functionality while improving code structure
* chore: refactor update debug query and disable more bidirectional stuff
* feat: init commit of Settings screen with navigation
- Add new Settings screen with sync and developer mode controls
- Set up React Navigation with TypeScript support
- Implement in-memory settings state management
- Add navigation between Home and Settings screens
Technical details:
- Configure react-native-screens and react-native-safe-area-context
- Add proper navigation types with RootStackParamList
- Fix WSL2/Windows development environment sync issues
- Work around react-native-screens minSdk version detection bug
- Remove AsyncStorage dependency temporarily for stability
This commit implements the Settings screen feature and sets up the
necessary
navigation infrastructure. The settings are currently managed in-memory
for
stability, with persistence to be implemented in a future commit.
* feat: add AsyncStorage persistence to Settings screen
- Implement persistent storage for app settings using AsyncStorage
- Add automatic loading of saved settings on component mount
- Add error handling for storage operations
- Maintain sync between AsyncStorage and ConfigObj state
- Use consistent storage key with app namespace
The settings screen now persists user preferences between app restarts,
including sync settings and developer mode toggle. Settings are loaded
automatically when the screen mounts and saved whenever they change.
* feat: move ConfigObj settings to UI with persistence
- Move Supabase and PowerSync configuration to UI
- Add validation to ensure all required settings are filled
- Update ConfigObj to reflect UI settings changes
- Disable sync toggle until all settings are valid
* fix: move from ConfigObj as settings truth to async storage
- Create useStoredSettings hook to manage settings state and persistence
- Replace ConfigObj usage with storedSettings throughout the app
- Add proper type safety with Settings interface
- Update Connector to use Settings interface instead of ConfigObj
- Add proper error handling and type safety in Connector
- Update navigation types to use proper NativeStackNavigationProp
- Improve UI feedback for sync state and settings configuration
- Add validation for settings before enabling sync
- Move PowerSync initialization to component level for better state
management
This change improves the app by:
1. Making settings persistent across app restarts
2. Providing better type safety throughout the codebase
3. Improving error handling and user feedback
4. Making the code more maintainable by centralizing settings
management
5. Following React best practices for state management
* chore: clean up UI
- Remove redundant header text and settings button from SetupSync screen
- Add menu button (three dots) to navigation bar for Settings access
- Add back button to Home screen that exits to main Android activity
- Adjust spacing between back button and title
- Update screen titles for better clarity ("Sync Info" and "Sync
Settings")
- Clean up commented out code in SetupSync component
* feat: move syncing to button for now
instead of on view load
* fix: update menu item name
* feat: make settings changes with a button
instead of on value change!
* fix: leave enable / disabled toggle seperate from other settings
* fix: settings across screens with settings provider
- Created new SettingsContext to manage global settings state
- Replaced useStoredSettings hook with context-based implementation
- Added SettingsProvider wrapper in App component
- Updated Settings, HomeScreen, and SetupSync components to use new context
- Removed old useStoredSettings.ts file
- Ensures settings changes are immediately reflected across all screens
- Maintains AsyncStorage persistence while improving runtime state management
* fix: disable dev menu for real build
* ci: hopefully faster builds
* ci: hopefully fix minsdk version stuff again
* ci: fix properties
* maint: upgrade gradle to latest supported by our react native version
https://github.com/facebook/react-native/blob/v0.74.5/gradle/wrapper/gradle-wrapper.properties
* ci: hopefully fix build
* feat(SetupSync): add PowerSync remote data management UI
- Add debug output section with selectable JSON display of sync status
- Add danger zone section for remote data management
- Add toggle button to show/hide danger zone
- Add warning message explaining remote-only deletion
- Add delete button for clearing remote PowerSync events
- Update button styling for better visual hierarchy
- Green for positive actions (sync)
- Blue for neutral actions (toggle)
- Red for dangerous actions (delete)
- Improve UX with clear warning messages and confirmations
- Add selectable text for easier debugging
The changes improve the debugging experience while adding safeguards
around dangerous operations that affect remote data.
* fix: hopefully build deps for release
* ci: try to fix release build again
* build: hopefully fix if caching was our problem
* fix: build by dropping safe area
can come back later after I figure out how to test the app to keep it
from breaking when I update things
* fix: safe area mock implementation to resolve React Navigation dependency issues
This commit addresses the dependency error caused by removing react-native-safe-area-context
while still using react-native-screens with React Navigation.
- Created a mock implementation of react-native-safe-area-context to satisfy
React Navigation's import requirements without the native code that was causing build issues
- Split the mock implementation into template files for easier review and maintenance:
* module-index.js - ES modules implementation
* commonjs-index.js - CommonJS implementation
* typescript-index.d.ts - TypeScript definition
* package.json - Package metadata
- Added setup script (setup_safe_area_mock.js) that generates the mock module from templates
- Added postinstall hook to package.json to automatically setup the mock after dependencies are installed
- Updated CI workflow to ensure the mock is properly set up before building
- Updated README with documentation about the mock implementation, its structure, and purpose
This is a temporary workaround until we can update SDK versions to properly
support both packages without breaking the build.
* feat: add scrollable view and collapsible debug panel to SetupSync
- Add ScrollView to ensure all content is accessible
- Implement toggleable debug section to keep UI elements visible
- Default debug panel to collapsed state for better UX
* feat: improve PowerSync connection state handling and UI feedback
- Add tri-state connection status logic (null|true|false) to handle initializing state
- Show informative blue banner during PowerSync initialization
- Only disable sync buttons when definitely disconnected, not during initialization
- Use PowerSync API's hasSynced property to properly detect initialization complete
- Add visual indicators to distinguish between initializing and disconnected states
- Prevent false disconnection warnings during PowerSync startup
* fix: missing Readme updates about mock
* fix: commmit missing fix to settings storage and better logging
* fix: settings page layout
* feat: env vars!
* maint: missing example
* Here's a summary for your commit:
Added support for arm64-v8a architecture required for Pixel 9 Pro and other modern Android devices. This includes:
1. **Native Library Support**
- Created script to build crsqlite.so for arm64-v8a architecture
- Added proper directory structure for ARM64 native libraries
2. **Build Configuration**
- Updated Android build.gradle to properly configure ABI filters
- Added APK splitting for optimized app size by architecture
- Fixed syntax in abiFilters declaration
3. **Documentation**
- Added detailed instructions for building native libraries for ARM64 devices
- Enhanced troubleshooting section with common issues and solutions
- Added specific instructions for Pixel 9 Pro support
4. **Toolchain Management**
- Added safeguards to ensure nightly Rust toolchain is used
- Documented issues with Homebrew Rust vs. official Rust installations
- Added validation for submodule initialization
- Full arm64-v8a support for native libraries
- Optimized APK size with architecture-specific builds
- Streamlined build process with new convenience scripts:
- `yarn build:crsqlite:arm64`
The changes ensure proper support for modern devices like Pixel 9 Pro while maintaining compatibility with x86_64 emulators and adding robust error handling and documentation for future developers.
* build: update GitHub Actions to support architecture-specific APKs
- Add support for split APKs by architecture (arm64-v8a, x86_64)
- Process both debug and release APKs for different architectures
- Implement clear naming convention for architecture-specific builds
- Update artifact uploads to include all architecture variants
- Enhance logging for better debugging and troubleshooting
- Update PR comments to indicate available architecture-specific builds
- Ensure GitHub releases include all architecture variants
* fix: build drop abifilters now that we split apks
* ci: bundle for debug
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.