`9.0.0-beta1 (2023-11-07) <https://github.com/neos/flow-development-collection/releases/tag/9.0.0-beta1>`_
==========================================================================================================

Overview of merged pull requests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

`FEATURE: Add more information for object arguments in debugging <https://github.com/neos/flow-development-collection/pull/3211>`_
----------------------------------------------------------------------------------------------------------------------------------

For stacktraces in exceptions and logs we now render some representation of content for objects to ease debugging with DTOs.

Specifically we will try to obtain a string representation for such an object by using either in this order:

- a string cast if __toString() is available
- json_encode if it is JsonSerializable
- json_encode on the array of public properties

For readability json_encode will be limited to the first level, also all of those string representations will be cut off after 100 characters.

If any of those options works we will also shorten the className to avoid this output becoming overly long.

Note that we use JSON_PARTIAL_OUTPUT_ON_ERROR to make sure some output is provided. This might lead to partial or weird outputs depending on the object structure, but might still provide pointers for debugging.

* Fixes: `#3165 <https://github.com/neos/flow-development-collection/issues/3165>`_

* Packages: ``Flow``

`9.0 FEATURE: Add `unique` flowQuery operation <https://github.com/neos/flow-development-collection/pull/3102>`_
----------------------------------------------------------------------------------------------------------------

This operation applies ``array_unique`` to the current flowQuery context.

While the same could previously achieved via ``Array.unique()`` the flow query operation can be placed in an operation chain without extra wrapping.

**Review instructions**

There is also a node specific implementation of the ``unique`` operation in https://github.com/neos/neos-development-collection/pull/4355

I know the php code looks oldish but the style is in line with the other flowQuery operations around. 


* Packages: ``Eel``

`FEATURE: Add `getAccessorByPath` to `Neos\Utility\Arrays` for type safe accessing of array values <https://github.com/neos/flow-development-collection/pull/3149>`_
--------------------------------------------------------------------------------------------------------------------------------------------------------------------

The array utility allows to create a type safe accessor via ``Arrays::getAccessorByPath($arrayValue, 'your.path')``. The accessor provides the following methods that will either return the requested type or throw a ``\\UnexpectedValueException``.  

* ``int(): int``
* ``float(): float``
* ``number(): int|float``
* ``string(): string``
* ``classString(): string`` - with annotation for class-string
* ``array(): array``
* ``instanceOf(string $className): object`` - with annotation for dynamic type
* ``intOrNull(): ?int``
* ``floatOrNull(): ?float``
* ``numberOrNull(): null|int|float``
* ``stringOrNull(): ?string``
* ``classStringOrNull(): ?string`` - with annotation for class-string | null
* ``arrayOrNull(): ?array``
* ``instanceOfOrNull(string $className): ?object`` - with annotation for dynamic type | null

This will allow to write code that accesses settings via pathes without checking every level for existence still beeing type safe and accessible for static analysis.

This can be used together with settingInjection.

```php
public function injectSettings(array $settings): void
{
   $this->limit = Arrays::getAccessorByPath($settings, 'limit')->intOrNull();
}
```

* Resolves: `#3164 <https://github.com/neos/flow-development-collection/issues/3164>`_

**Review instructions**

It may look inefficient to manually throw TypeErrors that in many cases would be thrown automatically because of the declared return types. However this is not a performance issue as those are never on the happy-path and the created TypeError provides additional informations to help understand and fix problems faster.


* Packages: ``Flow`` ``Utility.Arrays``

`FEATURE: Exclude classes from constructor autowiring <https://github.com/neos/flow-development-collection/pull/3070>`_
-----------------------------------------------------------------------------------------------------------------------

Classes can now explicitly be excluded from constructor autowiring through a new setting.

The setting accepts an array of fully qualified class names, each class name being a regular expression. Classes of scope prototype which expect objects to be passed to their constructor are usually considered for autowiring which results in a proxy class being generated.

This option allows to exclude classes from this process. This is useful for classes like data transfer objects, read models, commands, events and value objects which usually don't rely on dependency injection.

Flow cannot reliably detect weather a prototype class depends on autowiring for constructor arguments or not. Use this option to optimize your application to avoid the small but measurable overhead of proxy generation for those kinds of classes.

Note that if there are other reasons than constructor injection which require a proxy class to be generated, the proxy class will be generated no matter what.

This change partly reverts `#3050 <https://github.com/neos/flow-development-collection/issues/3050>`_because now proxy classes _are_ generated for prototype classes by default. Otherwise a lot of existing Flow applications would not work correctly anymore.

resolves: #3049

* Packages: ``Flow``

`FEATURE: Replace self with static in proxy classes <https://github.com/neos/flow-development-collection/pull/3074>`_
---------------------------------------------------------------------------------------------------------------------

Factory methods which use code like new self() for creating a new instance are now handled correctly in proxy classes. The compiler automatically replaces "self" keywords with "static" in the rendered proxy class file to make this possible.

This implementation has not been optimized for performance.

* Resolves: `#3059 <https://github.com/neos/flow-development-collection/issues/3059>`_

* Packages: ``Flow``

`FEATURE: Support private constructors in proxy classes <https://github.com/neos/flow-development-collection/pull/3072>`_
-------------------------------------------------------------------------------------------------------------------------

Flow now can correctly build proxy classes for classes with private constructors. Previously, such classes caused errors and proxy class building had to be disabled with the ``Proxy(false)`` annotation. Now classes with private constructors can take advantage of setter and property injection and are considered for advices through the AOP framework.

* Resolves: `#3058 <https://github.com/neos/flow-development-collection/issues/3058>`_

* Packages: ``Flow``

`FEATURE: Add support for readonly classes <https://github.com/neos/flow-development-collection/pull/3051>`_
------------------------------------------------------------------------------------------------------------

Flow now respects readonly classes during proxy class building and makes sure that proxy classes are readonly as well.

resolves: #3025

* Packages: ``Flow``

`BUGFIX: Use correct exception class <https://github.com/neos/flow-development-collection/pull/3209>`_
------------------------------------------------------------------------------------------------------

Fix the use of an exception class that is no longer where it was.


* Packages: ``Flow``

`BUGFIX: Use method to set validated instances container <https://github.com/neos/flow-development-collection/pull/3210>`_
--------------------------------------------------------------------------------------------------------------------------

* Fixes: `#3205 <https://github.com/neos/flow-development-collection/issues/3205>`_


* Packages: ``Flow``

`BUGFIX: Require collection packages as `self.version` again <https://github.com/neos/flow-development-collection/pull/3206>`_
------------------------------------------------------------------------------------------------------------------------------

* See: `#3035 <https://github.com/neos/flow-development-collection/issues/3035>`_ for the original change


* Packages: ``Flow`` ``Eel`` ``FluidAdaptor`` ``Kickstarter``

`BUGFIX: Only set distinct on count clause if explicitely set to improve performance <https://github.com/neos/flow-development-collection/pull/3140>`_
------------------------------------------------------------------------------------------------------------------------------------------------------

F.e. Postgres has performance issues with large datasets and the DISTINCT clause. In a test this change reduced the query time of a count query for ~900.000 entities by >80%.

In a custom project this affected their Neos Media.UI in which the following results were found:

* Count all assets |  580ms ->  260ms
* Query 20 assets  |  690ms ->  350ms
* Query 100 assets |  990ms -> 650ms
* Module load | 1900ms -> 1400ms

**Review instructions**

Everything should work the same, as https://github.com/neos/flow-development-collection/pull/415 already sets the distinct flag where (possibly) necessary.


* Packages: ``Flow``

`BUGFIX: Sanitize uploaded svg files from suspicious content <https://github.com/neos/flow-development-collection/pull/3172>`_
------------------------------------------------------------------------------------------------------------------------------

Adding an internal methods ``isSanitizingRequired`` and ``sanitizeImportedFileContent`` to the resourceManager. The import is adjusted to first determine the mediaType of an imported resource to decide wether sanitizing is needed which for now happens only for SVG files. If no sanitizing is needed the code will perform as before by passing streams or filenames around.

If suspicious content was removed from a warning is logged that mentions the remove data and line. The sanitizing is done using "enshrined/svg-sanitize" that is used by other cms aswell.

The initial implementation will only sanitize SVG files as those can contain malicious scripts. In future this should be expanded to a feature that allows registering of custom sanitizing functions.

The sanitizing logic itself ist basically the same as what is done by typo3 here: https://github.com/TYPO3/typo3/blob/`357b07064cf2c7f1735cfb8f73ac4a7248ab040e <https://github.com/neos/flow-development-collection/commit/357b07064cf2c7f1735cfb8f73ac4a7248ab040e>`_/typo3/sysext/core/Classes/Resource/Security/SvgSanitizer.php

This addresses the issue described here: https://nvd.nist.gov/vuln/detail/CVE-2023-37611

**Review Instructions**

The change adds quite a bit of complexity to the importResource method to avoid loading the file content into ram whenever possible. As this method accepts filenames and resources this leads to quite some nested checking. I consider this kindoff necessary as one does not want to read a full video file into php ram to check wether it may be an svg. 

Better suggestions are welcome.


* Packages: ``Utility.MediaTypes``

`BUGFIX: Replacement proxy methods rendered again <https://github.com/neos/flow-development-collection/pull/3101>`_
-------------------------------------------------------------------------------------------------------------------

This fixes a bug introduced in d939e6b8 switching to laminuas-code. A proxy method can replace the full body of an existing method or even be a fully new method, in which case only ``body`` will be set in the proxy method. We still want those to be generated. This for example currently breaks the CompileStatic feature, as those methods do not get rendered anymore resulting in worse performance in Production context compared to before.

This fix renders a proxy method also when a body was set for it, but still skips it if neither pre/post nor body is set.

It also enabled CompileStatic in Testing Context so that it is testable and adds a test to make sure it works as intended.

* Fixes: `#3099 <https://github.com/neos/flow-development-collection/issues/3099>`_

* Packages: ``Flow``

`BUGFIX: Remove injected properties before serialization <https://github.com/neos/flow-development-collection/pull/3067>`_
--------------------------------------------------------------------------------------------------------------------------

This fixes a regression introduced recently which resulted in serialization errors if the object to be serialized contained properties which were previously injected.

* Resolves: `#3066 <https://github.com/neos/flow-development-collection/issues/3066>`_

* Packages: ``Flow``

`BUGFIX: Support mixed return type in proxied methods <https://github.com/neos/flow-development-collection/pull/3065>`_
-----------------------------------------------------------------------------------------------------------------------

Flow's proxy class building now supports mixed return types for methods.

This change merely adds a test which proves that the feature is working. The actual implementation was part of https://github.com/neos/flow-development-collection/issues/3042.

resolves: https://github.com/neos/flow-development-collection/issues/2899

* Packages: ``Flow``

`BUGFIX: Union types in proxy classes <https://github.com/neos/flow-development-collection/pull/3057>`_
-------------------------------------------------------------------------------------------------------

Flow's proxy class building now supports union types in method signatures.

This change merely adds a test which proves that the feature is working. The actual implementation was part of #3042.

resolves: #2941

* Packages: ``Flow``

`BUGFIX: Create serialization code for transient properties <https://github.com/neos/flow-development-collection/pull/3063>`_
-----------------------------------------------------------------------------------------------------------------------------

Due to a recent optimization, Flow was not generating ``__sleep()`` methods for classes which are not either entities or were configured with a session scope. This led to errors in classes which were using the ``@Transient`` annotation to exclude certain properties from serialization. Therefore, Flow now also generates proxy classes with ``__sleep()`` methods if the original class contains such annotations.

* Resolves: `#3062 <https://github.com/neos/flow-development-collection/issues/3062>`_

* Packages: ``Flow``

`BUGFIX: Skip proxy for optional straight values <https://github.com/neos/flow-development-collection/pull/3052>`_
------------------------------------------------------------------------------------------------------------------

When a promoted property was an optional straight value, the proxy class builder decided to create a proxy class because it could be a straight value configured in the object configuration via Objects.yaml. Flow now checks the value of the given argument and only triggers proxy class building if the argument is not null. That way, Flow will not build useless proxies for typical read models which expect a mix of objects and straight values in their constructor.

related: `#1539 <https://github.com/neos/flow-development-collection/issues/1539>`_
related: `#3049 <https://github.com/neos/flow-development-collection/issues/3049>`_

* Packages: ``Flow``

`BUGFIX: Move access to objectAccess of TemplateObjectAccessInterface into getByPath <https://github.com/neos/flow-development-collection/pull/3041>`_
------------------------------------------------------------------------------------------------------------------------------------------------------

... as accessors are not used anymore for variable provider within fluid, starting v2.8.0.

Due to the missing accessors the ``objectAccess`` of ``TemplateObjectAccessInterface`` didn't get called anymore, so the result of the ``getByPath`` method was an object of ``FusionPathProxy`` instead of an rendered string.

See: 
https://github.com/TYPO3/Fluid/compare/2.7.4...2.8.0#diff-`a0aa72aa19d9eb57cdb9a4dcd344c3706d75ae7c <https://github.com/neos/flow-development-collection/commit/a0aa72aa19d9eb57cdb9a4dcd344c3706d75ae7c>`_a408286f91a846e495b3c766L122
https://github.com/TYPO3/Fluid/compare/2.7.4...2.8.0#diff-`a0aa72aa19d9eb57cdb9a4dcd344c3706d75ae7c <https://github.com/neos/flow-development-collection/commit/a0aa72aa19d9eb57cdb9a4dcd344c3706d75ae7c>`_a408286f91a846e495b3c766L341
https://github.com/TYPO3/Fluid/compare/2.7.4...2.8.0#diff-`a0aa72aa19d9eb57cdb9a4dcd344c3706d75ae7c <https://github.com/neos/flow-development-collection/commit/a0aa72aa19d9eb57cdb9a4dcd344c3706d75ae7c>`_a408286f91a846e495b3c766L312


* Packages: ``FluidAdaptor``

`!!! TASK: Modernized code style in ReflectionService <https://github.com/neos/flow-development-collection/pull/2914>`_
-----------------------------------------------------------------------------------------------------------------------

Code in the reflection service was adjusted to the current code style best practices. 

The method arguments in the Reflection Service are now strictly typed. Therefore, third-party code which relied on loose types and passes invalid types, need to be adjusted. Tests in the Flow package were adjusted were necessary.

As part of the clean up, the setStatusCache() method in ReflectionService was fixed which used a wrong order of parameters in its is_callable() call.

Preparation for #2913

* Packages: ``Flow``

`!!! TASK: Require PHP 8.2 <https://github.com/neos/flow-development-collection/pull/3040>`_
--------------------------------------------------------------------------------------------

The minimum requirement for the Flow Framework, including all packages of its distribution, was raised to PHP 8.2.

* Packages: ``Flow`` ``Utility.ObjectHandling``

`TASK: Migrate to PHPStan for Flow 8 <https://github.com/neos/flow-development-collection/pull/3218>`_
------------------------------------------------------------------------------------------------------

This is a backport of https://github.com/neos/flow-development-collection/pull/3216

Adds PHPStan level 1 to the whole Flow code base and CI.
Psalm was removed.


* Packages: ``Flow`` ``.github`` ``Cache``

`TASK: Carefully fix psalm types across codebase to make it green ;) <https://github.com/neos/flow-development-collection/pull/3199>`_
--------------------------------------------------------------------------------------------------------------------------------------

**Upgrade instructions**


* Packages: ``Flow``

`TASK: Update default settings for stored throwable dumps <https://github.com/neos/flow-development-collection/pull/3213>`_
---------------------------------------------------------------------------------------------------------------------------

This updates the default settings in YAML to 30 days of dump retention and a maximum of 10.000 files.

The class properties keep their ``0`` default, so that in case the class has been extended no change is enforced.

**Review instructions**

Needs upmerge of https://github.com/neos/flow-development-collection/pull/3187


`TASK: Use new Behat `FlowBootstrapTrait` <https://github.com/neos/flow-development-collection/pull/3208>`_
-----------------------------------------------------------------------------------------------------------

Adjust to behat adjustments see https://github.com/neos/behat/pull/35

**Upgrade instructions**


* Packages: ``Flow``

`TASK: Clean up stored throwable dumps <https://github.com/neos/flow-development-collection/pull/3187>`_
--------------------------------------------------------------------------------------------------------

Whenever a new dump is written, check the existing dumps and remove those that are older than allowed or exceed the maximum count.

By default nothing is cleaned up.

* Resolves: `#3158 <https://github.com/neos/flow-development-collection/issues/3158>`_

**Review instructions**

Should remove old dump files as configured…


* Packages: ``Flow``

`TASK: Fix overlooked dependency… <https://github.com/neos/flow-development-collection/pull/3207>`_
-----------------------------------------------------------------------------------------------------

* See: `#3035 <https://github.com/neos/flow-development-collection/issues/3035>`_ for the original change


* Packages: ``Flow``

`TASK: Fix cache RedisBackend unittest <https://github.com/neos/flow-development-collection/pull/3196>`_
--------------------------------------------------------------------------------------------------------

A test failed due to a missing return value from a method not being mocked (correctly),


* Packages: ``Cache``

`TASK: Fix documentation builds <https://github.com/neos/flow-development-collection/pull/3195>`_
-------------------------------------------------------------------------------------------------

… by pinning updated dependencies.

**Review instructions**

Best is to see if the builds succeed on RTD again with this merged…


* Packages: ``Flow``

`TASK: document and deprecate flows internal isolated behat tests <https://github.com/neos/flow-development-collection/pull/3173>`_
-----------------------------------------------------------------------------------------------------------------------------------

Related https://github.com/neos/flow-development-collection/issues/3170

The infrastructure is quite complex and not in relation to those two tests. That's why we declare it ready to be removed.

**Upgrade instructions**


* Packages: ``Flow``

`TASK: Support PHP never, null, false, and true as stand-alone types <https://github.com/neos/flow-development-collection/pull/3071>`_
--------------------------------------------------------------------------------------------------------------------------------------

This change adds functional tests to prove that Flow can handle PHP 8 stand-alone return types in AOP proxy class building.

Note that "null" is not supported yet by laminas-code, therefore the corresponding test is not active yet.

* Resolves: `#3027 <https://github.com/neos/flow-development-collection/issues/3027>`_

* Packages: ``Flow``

`TASK: Use Laminas Code for proxy method rendering <https://github.com/neos/flow-development-collection/pull/3064>`_
--------------------------------------------------------------------------------------------------------------------

Flow now uses laminas/laminas-code for rendering proxy methods. The Dependency Injection Proxy Class Builder was refactored and the classes ProxyConstructor and ProxyMethod were replaced by new implementations called ProxyConstructorGenerator and ProxyMethodGenerator respectively.

* Resolves: `#3042 <https://github.com/neos/flow-development-collection/issues/3042>`_

* Packages: ``Flow``

`TASK: Clean up code in AOP and ObjectManagement <https://github.com/neos/flow-development-collection/pull/3055>`_
------------------------------------------------------------------------------------------------------------------

This change contains various code clean-ups which fell off with the preparation of a new bug fix for AOP.

* Packages: ``Flow``

`TASK: Replace "adviced" by "advised" <https://github.com/neos/flow-development-collection/pull/3054>`_
-------------------------------------------------------------------------------------------------------

Fixed a good old typo everywhere in Flow by replacing all occurrences of "adviced" by "advised".

* Packages: ``Flow``

`TASK: Clean up functional tests for AOP <https://github.com/neos/flow-development-collection/pull/3053>`_
----------------------------------------------------------------------------------------------------------

This also re-activates a functional test targeting PHP 7.1 features which was disabled at some point in history.

* Packages: ``Flow``

`TASK: Only add constructor injection code if needed <https://github.com/neos/flow-development-collection/pull/3050>`_
----------------------------------------------------------------------------------------------------------------------

The proxy class builder now skips code generation for constructor injection code if the given original class is prototype, no user-defined object configuration exists and all potentially autowired constructor arguments are prototypes or simple values. This change should result in a significantly less amount of proxy classes generated in most modern Flow projects.

resolves: `#3049 <https://github.com/neos/flow-development-collection/issues/3049>`_
resolves: #1539

* Packages: ``Flow``

`TASK: Only add serialization entities code if needed <https://github.com/neos/flow-development-collection/pull/3047>`_
-----------------------------------------------------------------------------------------------------------------------

Proxy classes created by the Dependency Injection Proxy Class Builder now only contain code related to serialization and deserialization of related entities if needed.

The code is only rendered if one of the following conditions is met:

- The class is annotated with Entity
- The class is annotated with Scope("session")

Despite the previous condition, the code will not be rendered if the following condition is true:

- The class already has a __sleep() method (we assume that the developer wants to take care of serialization themself)

As part of this change, the generated code related to serialization was slightly adjusted for stricter type handling.

related: `#1539 <https://github.com/neos/flow-development-collection/issues/1539>`_

**Review instructions**

- try to find an existing application which relies on serialization of related entities, for example a Flow application which uses ORM with relations or uses entities in a session scope.
- remove all caches and then access your application in a browser using the current Flow 9 branch (without this patch)
- create a backup of the Cache/Code/Flow_Object_Classes directory
- switch to a branch with this change, remove all caches and access the application again in a browser
- use a diff tool (e.g. Kaleidoscope) to compare both cache directories to see what is now different
- check if your application still works

* Packages: ``Flow``

`Detailed log <https://github.com/neos/flow-development-collection/compare/8.3.5...9.0.0-beta1>`_
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
