* implement disk state manager
* put states inside of a folder
* return root state all the time
* factor out code
* add docs for token expiration
* cache states directory
* call absolute on web directory
* change dir to app path when testing the backend
* remove accidental 🥒
* test disk for now
* modify schema
* only serialize specific stuff
* fix issue in types
* what is a kilometer
* create folder if it doesn't exist in write
* this code hates me
* check if the file isn't empty
* add try except clause
* add check for directory again
* fully migrate vars into new system
* i hate rufffff (no i don't)
* fix silly pright issues (except colormode and state)
* remove all instances of Var.create
* create immutable callable var and get rid of more base vars
* implement hash for all functions
* get reflex-web to compile
* get it to compile reflex-web successfully
* fix tests
* fix pyi
* use override from typing_extension
* put plotly inside of a catch
* dicts are unusable sadly
* fix silly mistake
* overload equals to special case immutable var
* improve test_cond
* solve more CI issues, down to 94 failures
* down to 20 errors
* down to 13 errors
* pass all testcases
* fix pyright issues
* reorder things
* use get origin more
* use fixed_type logic
* various optimizations
* go back to passing test cases
* use less boilerplate
* remove unnecessary print message
* remove weird comment
* add test for html issue
* add type ignore
* fix another silly issue
* override get all var data for var operations call
* make integration tests pass
* fix immutable call var
* better logic for finding parent class
* use even better logic for finding state wrt computedvar
* only choose the ones that are defined in the same module
* small dict to large dict
* [REF-3591] Remove chakra-related files from immutable vars PR (#3821)
* Add comments to html metadata component (#3731)
* fix: add verification for path /404 (#3723)
Co-authored-by: coolstorm <manas.gupta@fampay.in>
* Use the new state name when setting `is_hydrated` to false (#3738)
* Use `._is_mutable()` to account for parent state proxy (#3739)
When a parent state proxy is set, also allow child StateProxy._self_mutable to
override the parent's `_is_mutable()`.
* bump to 0.5.9 (#3746)
* add message when installing requirements.txt is needed for chosen template during init (#3750)
* #3752 bugfix add domain for XAxis (#3764)
* fix appharness app_source typing (#3777)
* fix import clash between connectionToaster and hooks.useState (#3749)
* use different registry when in china, fixes#3700 (#3702)
* do not reload compilation if using local app in AppHarness (#3790)
* do not reload if using local app
* Update reflex/testing.py
Co-authored-by: Masen Furer <m_github@0x26.net>
---------
Co-authored-by: Masen Furer <m_github@0x26.net>
* Bump memory on relevant actions (#3781)
Co-authored-by: Alek Petuskey <alekpetuskey@Aleks-MacBook-Pro.local>
* [REF-3334] Validate Toast Props (#3793)
* [REF-3536][REF-3537][REF-3541] Move chakra components into its repo(reflex-chakra) (#3798)
* fix get_uuid_string_var (#3795)
* minor State cleanup (#3768)
* Fix code wrap in markdown (#3755)
---------
Co-authored-by: Alek Petuskey <alek@pynecone.io>
Co-authored-by: Manas Gupta <53006261+Manas1820@users.noreply.github.com>
Co-authored-by: coolstorm <manas.gupta@fampay.in>
Co-authored-by: Thomas Brandého <thomas.brandeho@gmail.com>
Co-authored-by: Shubhankar Dimri <dimrishubhi@gmail.com>
Co-authored-by: benedikt-bartscher <31854409+benedikt-bartscher@users.noreply.github.com>
Co-authored-by: Khaleel Al-Adhami <khaleel.aladhami@gmail.com>
Co-authored-by: Alek Petuskey <alekpetuskey@Aleks-MacBook-Pro.local>
Co-authored-by: Elijah Ahianyo <elijahahianyo@gmail.com>
* pyproject.toml: bump to 0.6.0a1
* pyproject.toml: depend on reflex-chakra>=0.6.0a
New Var system support in reflex-chakra 0.6.0a1
* poetry.lock: relock dependencies
* integration: bump listening timeout to 1200 seconds
* integration: bump listening timeout to 1800 seconds
* Use cached_var_no_lock to avoid ImmutableVar deadlocks (#3835)
* Use cached_var_no_lock to avoid ImmutableVar deadlocks
ImmutableVar subclasses will always return the same value for a _var_name or
_get_all_var_data so there is no need to use a per-class lock to protect a
cached attribute on an instance, and doing so actually is observed to cause
deadlocks when a particular _cached_var_name creates new LiteralVar instances
and attempts to serialize them.
* remove unused module global
---------
Co-authored-by: Masen Furer <m_github@0x26.net>
Co-authored-by: Alek Petuskey <alek@pynecone.io>
Co-authored-by: Manas Gupta <53006261+Manas1820@users.noreply.github.com>
Co-authored-by: coolstorm <manas.gupta@fampay.in>
Co-authored-by: Thomas Brandého <thomas.brandeho@gmail.com>
Co-authored-by: Shubhankar Dimri <dimrishubhi@gmail.com>
Co-authored-by: benedikt-bartscher <31854409+benedikt-bartscher@users.noreply.github.com>
Co-authored-by: Alek Petuskey <alekpetuskey@Aleks-MacBook-Pro.local>
Co-authored-by: Elijah Ahianyo <elijahahianyo@gmail.com>
* [REF-3184] Raise exception when encountering nested `async with self` blocks
Avoid deadlock when the background task already holds the mutation lock for a
given state.
* [REF-3339] get_state from background task links to StateProxy
When calling `get_state` from a background task, the resulting state instance
is wrapped in a StateProxy that is bound to the original StateProxy and shares
the same async context, lock, and mutability flag.
* If StateProxy has a _self_parent_state_proxy, retrieve the correct substate
* test_state fixup
* add module prefix to state names
* fix state names in test_app
* update state names in test_state
* fix state names in test_var
* fix state name in test_component
* fix state names in test_format
* fix state names in test_foreach
* fix state names in test_cond
* fix state names in test_datatable
* fix state names in test_colors
* fix state names in test_script
* fix state names in test_match
* fix state name in event1 fixture
* fix pyright and darglint
* fix state names in state_tree
* fix state names in redis only test
* fix state names in test_client_storage
* fix state name in js template
* add `get_state_name` and `get_full_state_name` helpers for `AppHarness`
* fix state names in test_dynamic_routes
* use new state name helpers in test_client_storage
* fix state names in test_event_actions
* fix state names in test_event_chain
* fix state names in test_upload
* fix state name in test_login_flow
* fix state names in test_input
* fix state names in test_form_submit
* ruff
* validate state module names
* wtf is going on here?
* remove comments leftover from refactoring
* adjust new test_add_style_embedded_vars
* fix state name in state.js
* fix integration/test_client_state.py
new SessionStorage feature was added with more full state names that need to be formatted in
* fix pre-commit issues in test_client_storage.py
* adjust test_computed_vars
* adjust safe-guards
* fix redis tests with new exception state
---------
Co-authored-by: Masen Furer <m_github@0x26.net>
* add computed backend vars
* finish computed backend vars, add tests
* fix token for AppHarness with redis state manager
* fix timing issues
* add unit tests for computed backend vars
* automagically mark cvs with _ prefix as backend var
* fully migrate backend computed vars
* rename is_backend_variable to is_backend_base_variable
* add integration test for implicit backend cv, adjust comments
* replace expensive backend var check at runtime
* keep stuff together
* simplify backend var check method, consistent naming, improve test typing
* fix: do not convert properties to cvs
* add test for property
* fix cached_properties with _ prefix in state cls
* test_background_task: dispatch multiple async tasks
Use asyncio.gather to dispatch multiple tasks from a single background task
that all compete over the `async with self` lock. Even though the state itself
has a lock, each StateProxy instance should only allow a single `async with
self` context to run at a time.
* Protect StateProxy with an asyncio.Lock
Allow multiple tasks to reference the same StateProxy without stomping on each
other when entering an `async with self` context to acquire the state lock and
ultimately modify the state.
Always treat ComponentState subclasses as mixin, and explicitly pass
`mixin=False` in `.create` classmethod when intentionally creating a substate.
This allows a "base" ComponentState to have subclasses that also work as
ComponentState themselves.
Fix#3368
* typed mixins
* implicit mixin=True kwarg for ComponentState subclasses
* fix: always init other subclasses
* adjust tests: all mixins support base vars now
* upgrade to latest pip for in_docker_test_script.sh
* Bump gunicorn to 22.0.0 (security)
Changelog: https://docs.gunicorn.org/en/stable/news.html#id1
use utime to notify workers liveness
migrate setup to pyproject.toml
fix numerous security vulnerabilities in HTTP parser (closing some request smuggling vectors)
parsing additional requests is no longer attempted past unsupported request framing
on HTTP versions < 1.1 support for chunked transfer is refused (only used in exploits)
requests conflicting configured or passed SCRIPT_NAME now produce a verbose error
Trailer fields are no longer inspected for headers indicating secure scheme
support Python 3.12
** Breaking changes **
minimum version is Python 3.7
the limitations on valid characters in the HTTP method have been bounded to Internet Standards
requests specifying unsupported transfer coding (order) are refused by default (rare)
HTTP methods are no longer casefolded by default (IANA method registry contains none affected)
HTTP methods containing the number sign (#) are no longer accepted by default (rare)
HTTP versions < 1.0 or >= 2.0 are no longer accepted by default (rare, only HTTP/1.1 is supported)
HTTP versions consisting of multiple digits or containing a prefix/suffix are no longer accepted
HTTP header field names Gunicorn cannot safely map to variables are silently dropped, as in other software
HTTP headers with empty field name are refused by default (no legitimate use cases, used in exploits)
requests with both Transfer-Encoding and Content-Length are refused by default (such a message might indicate an attempt to perform request smuggling)
empty transfer codings are no longer permitted (reportedly seen with really old & broken proxies)
** SECURITY **
fix CVE-2024-1135
* Remove TYPE_CHECKING guard for pydantic v1 imports
Retain TYPE_CHECKING guard in v1 fallback to force pyright into pydantic.v1 namespace
* Run unit tests with pydantic v1 now that v2 is installed via poetry
* Allow EventHandler args to be partially applied
When an EventHandler is called with an incomplete set of args it creates a
partial EventSpec. This change allows Component._create_event_chain to apply
remaining args from an args_spec to an existing EventSpec to make it
functional.
Instead of requiring the use of `lambda` functions to pass arguments to an
EventHandler, they can now be passed directly and any remaining args defined in
the event trigger will be applied after those.
* [REF-2273] Implement `.setvar` special EventHandler
All State subclasses will now have a special `setvar` EventHandler which
appears in the autocomplete drop down, passes static analysis, and canbe used
to set State Vars in response to event triggers.
Before:
rx.input(value=State.a, on_change=State.set_a)
After:
rx.input(value=State.a, on_change=State.setvar("a"))
This reduces the "magic" because `setvar` is statically defined on all State
subclasses.
* Catch invalid Var names and types at compile time
* Add test cases for State.setvar
* Use a proper redis-compatible token
* Add test_get_state_from_sibling_not_cached
A better unit test to catch issues with refetching parent states
and calculating the wrong parent state names to fetch.
* _determine_missing_parent_states: correctly generate state names
Prepend only the previous state name to the current relative_parent_state_name
instead of joining all of the previous state names together.
* [REF-2219] Avoid refetching states that are already cached
The already cached states may have unsaved changes which can be wiped out if
they are refetched from redis in the middle of handling an event.
If the root state already knows about one of the potentially missing states,
then use the instance that is already cached.
Fix#2851
* Connection pulser only depends on has_connection_errors
Avoid showing the WiFi error icon when the state is hydrating / navigating
because not being hydrated is not indicative of a connection error in itself.
* Set is_hydrated=False at route onChangeStart
When navigation event starts, set is_hydrated=False on the client side before
any on_load event is dispatched. This avoids a flickering problem where the
client browser navigates and briefly shows content on the page before
processing on_load events associated with the page.
Fix#2885
* Update pyi
* Remove upper bounds of most dependencies.
Also adds a import try except block for pydantic.v1 and relocks.
Keep black and ruff to not mess to much with current formatting
Make pyright see the right import as long as constraint still lock pydantiv v1
Down pin pytest-asyncio again due to known issue
Fix upload handler with latest versions of fastapi
Change comment
* Add changed lockfile
* Set max versions for deps
* Revert app.pyi
---------
Co-authored-by: Malte Klemm <malte.klemm@blueyonder.com>
Co-authored-by: Nikhil Rao <nikhil@reflex.dev>
* [REF-2265] ComponentState: scaffold for copying State per Component instance
Define a base ComponentState which can be used to easily create copies of the
given State definition (Vars and EventHandlers) that are tied to a particular
instance of a Component (returned by get_component)
* Define `State` field on `Component` for typing compatibility.
This is an Optional field of Type[State] and is populated by ComponentState.
* Add integration/test_component_state.py
Create two independent counters and increment them separately
* Add unit test for ComponentState
* Use dill instead of cloudpickle for serialization
* smaller size pickles
* support dynamically defined states
* avoid issues with unpickleable globals
* pyproject: add dill, remove cloudpickle
* poetry.lock: relock dependencies
* Dynamically convert EventHandler to functools.partial
Instead of converting the functions up front and assigning them to the
instance, unbox the function from the EventHandler when it is requested via
__getattribute__. This reduces the size of the per-instance pickle, because
event handler bodies do not need to be included.
* Improve checking for cython_function_or_method
Because pydantic can be installed without cython, only use the workaround in
the case where the BaseModel.validate function is NOT a FunctionType,
indicating it's a cython function.
* Serialize all State subclasses by reference
* rebase
* pass include_children kwarg in radix FormRoot
* respect include_children
* ruff fixes
* readd statemanager init, run pyi gen
* minor performance imporovements, fix for state changes
* fix pyi and pyright
* pass include_children for chakra
* remove old state detection
* add test for unused states in stateless app
---------
Co-authored-by: Masen Furer <m_github@0x26.net>
* test_potentially_dirty_substates: when a state named State should be computed
Catch a regression introduced in 0.4.3a1
* _potentially_dirty_substates: qualify substate name
When looking up substate classes, ensure the qualified name is used to avoid
issues with same-named substates.
* initial attempt at writing test for urls
* turn it into a pytest test
* fix precommit and add wrong url to make sure test work
* fix darglint error and fix url set error
* black error
* add to test .md files in reflex as well
* update to fix masen comment
* black formatting
* Fix trailing slash in reflex/state.py
---------
Co-authored-by: Tom Gotsman <tomgotsman@Toms-MacBook-Pro.local>
Co-authored-by: Tom Gotsman <tomgotsman@toms-mbp.lan>
Co-authored-by: Masen Furer <m_github@0x26.net>
* added code for computed backend vars
* fixed formatting issues
* fix small bug
* fixes ruff issue
* fixed black issue
* augment test for backend computed var
---------
Co-authored-by: Masen Furer <m_github@0x26.net>
* WiP get_state
* Refactor get_state fast path
Rudimentary protection for state instance access from a background task
(StateProxy)
* retain dirty substate marking per `_mark_dirty` call to avoid test changes
* Find common ancestor by part instead of by character
Fix StateProxy for substates and parent_state attributes (have to handle in
__getattr__, not property)
Fix type annotation for `get_state`
* test_state: workflow test for `get_state` functionality
* Do not reset _always_dirty_substates when adding vars
Reset the substate tracking only when the class is instantiated.
* test_state_tree: test substate access in a larger state tree
Ensure that `get_state` returns the proper "branch" of the state tree depending
on what substate is requested.
* test_format: fixup broken tests from adding substates of TestState
* Fix flaky integration tests with more polling
* AppHarness: reset _always_dirty_substates on rx.State
* RuntimeError unless State is instantiated with _reflex_internal_init=True
Avoid user errors trying to directly instantiate State classes
* Helper functions for _substate_key and _split_substate_key
Unify the implementation of generating and decoding the token + state name
format used for redis state sharding.
* StateManagerRedis: use create_task in get_state and set_state
read and write substates concurrently (allow redis to shine)
* test_state_inheritance: use polling cuz life too short for flaky tests
kthnxbai ❤️
* Move _is_testing_env to reflex.utils.exec.is_testing_env
Reuse the code in app.py
* Break up `BaseState.get_state` and friends into separate methods
* Add test case for pre-fetching cached var dependency
* Move on_load_internal and update_vars_internal to substates
Avoid loading the entire state tree to process these common internal events. If
the state tree is very large, this allow page navigation to occur more quickly.
Pre-fetch substates that contain cached vars, as they may need to be recomputed
if certain vars change.
* Do not copy ROUTER_DATA into all substates.
This is a waste of time and memory, and can be handled via a special case in
__getattribute__
* Track whether State instance _was_touched
Avoid wasting time serializing states that have no modifications
* Do not persist states in `StateManagerRedis.get_state`
Wait until the state is actually modified, and then persist it as part of `set_state`.
Factor out common logic into helper methods for readability and to reduce
duplication of common logic.
To avoid having to recursively call `get_state`, which would require persisting
the instance and then getting it again, some of the initialization logic
regarding parent_state and substates is duplicated when creating a new
instance. This is for performance reasons.
* Remove stray print()
* context.js.jinja2: fix check for empty local storage / cookie vars
* Add comments for onLoadInternalEvent and initialEvents
* nit: typo
* split _get_was_touched into _update_was_touched
Improve clarity in cases where _get_was_touched was being called for its side
effects only.
* Remove extraneous information from incorrect State instantiation error
* Update missing redis exception message
* fix inherited backend var deps
* fixing the real issue with @masenf
* Revert "fix inherited backend var deps"
This reverts commit 894e3e14ac.
* python 3.8
* initial values for computed vars draft
* add tests, add computed_var overloads
* fix darglint
* pass initial to substates when calling dict
* add tests for for child states
* format black
* allow None as initial value
* rename runtime_only to raises_at_runtime
* cleanup unused arguments of ComputedVars
* refactor cached_var to be partial of computed_var
* test_state: augment modify_state test for writing MutableProxy
If the object contains a MutableProxy inside of it, then we get a pickling
error.
* Implement __reduce_ex__ for MutableProxy
Pass through `__reduce_ex__` onto the wrapped instance to strip it off when
cloudpickling to redis.
* base: get_value actually works with a str key
Unless the key isn't a field on the model, then it falls back to the previous
behavior of just returning the given key as is... why does it do this? I don't
know.
* Move sharding internal to StateManager
Avoid leaking sharding implementation details all over the State class and
breaking the API
* WiP StateManager based sharding
* Copy the state __dict__ when serializing to avoid breaking the instance
* State tests need to pass the correct substate token for redis
* state: when getting parent_state, set top_level=False
ensure that we don't end up with a broken tree
* test_app: get tests passing with redis by passing the correct token
refactor upload tests to suck less
* test_client_storage: look up substate key
* state.py: pass static checks
* test_dynamic_routes: working with redis state shard
* Update the remaining AppHarness tests to pass {token}_{state.get_full_name()}
* test_app: pass all tokens with state suffix
* StateManagerRedis: clean up commentary
* feat: Synchronizing localStorage between tabs using browser events
* test_client_storage: Test sync'd local storage vars
* update_vars_internal: generic handler to apply var changes to state tree
Apply fully qualified var names to each substate they are associated with. This
allows consistent updates to arbitrary state vars without having to know their
"setter" arguments, in case the user has overwritted the `set_x` name.
---------
Co-authored-by: Masen Furer <m_github@0x26.net>
* wip event handlers for state mixins
* fix computed vars, prepare event handler tests
* fix computed vars and event handlers inherited by mixins
* Base is not a mixin
* improve state inheritance tests
* wait for alerts to appear
* poll_for_content to fix flaky ci tests
* use redis-py url syntax for redis_url
* port is optional
* Add StateManagerRedis.close method
The close helper method always calls `close_connection_pool=True` so that all
outstanding redis operations can be stopped before changing event loops.
---------
Co-authored-by: Masen Furer <m_github@0x26.net>
* test_client_storage: remove race conditions for cookie assignment
Poll for default timeout for cookies to appear in the controlled browser.
* Remove use of deprecated get_token and get_sid in core
Both reflex.app and reflex.state were still using deprecated methods, which
were throwing unsolvable warnings for end users.
* Remove deprecated router functions from integration tests
Mostly removing custom "token" var and replacing with
router.session.client_token.
Also replacing `get_query_params` and `get_current_page` usage as well.
* fix upload tests
Cannot pass substate as main app state, since it blocks us from accessing
"inherited vars"
* state: do NOT reset `router` to default
When calling `.reset` to reset state vars, do NOT reset the router data, as
that could mess up internal event processing.
