Symfony World Online
Because of the current situation in the world, Symfony events are also digital this year. The Symfony community has chosen Hopin as a tool to give Symfony World. I am already a fan of the simplicity of this tool to quickly choose and follow a session.
It has been a few years since I used a Symfony application myself. In the meantime I only worked with Drupal or Laravel. So you will see further on in these notes that I relate to these frameworks to make things clearer for myself as well.
In this keynote Fabien talked about the great gamechangers of Symfony:
- The Symfony Community, many one-time contributors but also a dedicated docs (7 people) and core team (17 people).
- HttpKernelInterface (from symphony2 - not adapted for 10 years) based on http standards and also used by many other frameworks. (Drupal also uses this)
- Symfony Flex, the glue code that throws packages together to the full stack framework. This ensures that php libraries can easily be used as first-class citizens. Flex is an enabler for faster application development (RAD). Then came a demo of the Symfony command that uses the creator bundle for quick setup and class-generation, as well as integration of docker & docker-compose.
After that he discussed what the next game changer for Symfony could be.
One of the examples shown was to use dropzone in a Symfony form. After installing the package this was just changing the input type.
This seems to me a better solution than asset-packagist that is often used in the Drupal community.
Additional integrations have been added: https://stimulusjs.org/, in collaboration with Swup to enable a more SPA-like way of page loading. This is based on turbolinks and very similar to refreshless in the Drupal world.
Blurhash placeholder images have also been added to allow a faster image display. This ensures a much better UX for slower connections or pages with a large number of images. I think this is a very interesting library that we will definitely use in our own projects.
See also Fabien's slides here: https://speakerdeck.com/fabpot/symfony-ux.
Decoupling an application with Symfony Messenger
In Symfony applications, this is very similar to an event-based system - but over different threads and not the standard PHP single-threaded way of working. Of course, PHP can only work over one thread at a time, which is why it is effectively handled by a different process. The original process stops when the message is delivered to the bus.
One of the big advantages of a message is that it is asynchronous, the sender does not wait for a response from the receiver. Another advantage is that they are reliable, when the processor stops answering, the message waits in the bus until it can be picked up.
These are the major changes compared to the Event system.
- Better application structure because not everything has to be in the same place, this is partly also an advantage of the event system. But there everything still has to live in the same application. A message that comes via a bus should not be handled in the same application and not even in the same language.
- Asynchronous processing of data.
- It is an easy way to increase scale. Multiple workers can be added to handle heavy events. This way it is easy to scale up where possible.
Furthermore David goes very deep into how the Symfony Messenger works. But there are also a number of framework diagnostic tips, including this one:
Your message is quite small and only contains the id of the object it's about. The subscriber will then retrieve the same data from the database. This ensures that the receiver always has the most recent data. This tip was contradicted in the next session of Samuel Roze, according to him it is sometimes possible to put all the data in the message. Interesting that it is always looked at from our own background.
Symfony Messenger also has rejuvenation built-in. This means that a process itself can process a number of messages before it closes itself. This process can then be restarted by supervisor/docker.
The next concept was about transport decorators, this is a way to wrap a message that is sent out with extra metadata before it is effectively put on the bus.
David gave some examples, including deduplicate - this is a semaphore system in Redis. This ensures that a product can only be put on the queue once until it is finished. This way, fast updates can't overload the queue.
In the chat of the talk there was a lot of talk about message architectures. A tip was to use the doctrine/database transport by default, which is easy to start with.
There are other implementations possible as soon as there is scaling up, moving your messages to Amazon SQS, Redis, AMQP or Apache Kafka will automatically reduce the load on your database.
Symfony meets Cypress - E2E testing for symfony developers
End-to-end tests are sometimes described as worfklow testing or integration testing. This is where a computer simulates a real user and really clicks, navigates, types.
However, there are a number of pain points associated with writing end-to-end tests.
- They are slow to run compared to unit tests.
- Difficult to debug.
- Difficult to maintain
- Tests are sometimes flaky, this can also occur with other types of tests (such as unit tests) but is more painful because of the other difficulties.
Why use Cypress to write e2e tests? It already offers 2 solutions of the above list, they are quick and easy to debug.
- Cypress is an all-in-one tool.
- Tests are written in es6-js.
- UI (testrunner) with insights directly included.
- No selenium/webdriver needed on the system running the tests.
- Console/network tab available during tests, because they can run in the user's browser.
Ramona also gave a live demo of how this works. This demo went from creating the test directory to running the test and writing a few assertions, as this was only part of the talk, I am impressed with the speed with which the first test could be written.
Cypress can use selectors to select the right items, the Cypress UI has a tool that can help to find the right selector. Ids can be generated, so you have to be careful because they are not always the same.
The debugging features of the test runner in the browser are very interactive and you can see the history of a test immediately. This way you can clearly see what goes wrong, I think this is really a huge advantage compared to testing with Behat.
Spec tests have similar syntax as JEST unit tests. They use the describe-it (Mocha) syntax to write tests. In PHP we also have the Kahlan framework that uses this syntax. Also the new syntax of pestphp is similar. So it is a good thing that an existing way of writing tests has been chosen here. This makes it easier to find documentation, clearly a plus in my opinion.
There are also docker images created by the Cypress team. These offer an easy way to start testing in cypress: https://www.cypress.io/blog/2019/05/02/run-cypress-with-a-single-docker-command/.
We are definitely going to test this method of testing.
Streaming: an alternative to CRUD & batch processing
This session was about how to discuss the consequences and challenges of event streaming in PHP and the choice of architecture for this.
Events in this context are messages that run in real-time between different services/components of a system. This can be the same application or different components of a microservice distributed system.
Data that goes to one of these services is about a queue, and when you make such a queue persistent, something like this can continue to work if the service that is going to process the message is temporarily offline.
This processing service doesn't need to know we sent the event, it just needs to know what an event of the type it needs to process looks like, with the obvious advantage of clearer boundaries throughout your application.
There are a number of major downsides to this way of working:
- Writing code for distrubuted systems is hard.
- You need governance about how message bus is used.
- You need to be sure that a message has been sent, a solution for this can be the "outbox pattern", or always save your queue in a table. The outbox saves the data in a local table and in a separate outbox table. This way we know for sure that saving the message and the real data is successful.
- Guaranteed exactly once delivery is very difficult. Messages can sometimes be picked up by 2 different workers at the same time. The problem if this is not followed up properly: event 2x processed so a product is purchased 2x for example.
According to Samuel it can happen that you get the same action more than once, you have to take this into account when building your system. You can do this by using the same actions, if you perform an action more than once with the same input the same output should come out every time. A solution is to include the current state in the message or an idempotency key. This key represents a message and so you can see if it has already been processed.
- Processing messages in parallel:
This can greatly improve the speed of the application, by starting a new worker you can double the effective bandwidth of your application. Be careful with this as it can cause us to lose state.
The easiest solution is not competing processes, the other solution is locking. This way you lock an entity so it can only be processed by one worker. The second worker can then continue with another entity. Such an optimistic locking can be done with Symfony Lock (See also Lock & Semaphore).
I will have to look at this talk a few more times to understand everything that has been explained. Samuel tried very hard to explain everything clearly - but there was a lot of information in it. In any case, I can already see a lot of practical applications for this.
Lock & Semaphore: The gatekeepers of your resources
This talk is about two Symfony components that are very related to each other.
- symphony/lock (available since sf 3.4)
- symphony/semaphore (available since sf 5.2)
A Semaphore allows N processes to process a resource, and does not give that possibility to the rest, a Lock is a Semaphore where N equals 1.
In this talk a number of features of both components were discussed:
- The same key instance is needed to unlock a lock back. This key can be serialized - but it must be the same.
- There is an Auto release functionality, this happens when a lock is garbage-collected, that's why you always have to assign your lock to a variable at acquire, otherwise it can be removed when you exit that function.
- Blocking mode: acquire(true) - guaranteed that lock is acquired by this process.
- Expiration: to prevent zombie locks, it's not easy to estimate a few seconds here, you can use lock->refresh, that way the lock is not too long - but it will be refreshed in a loop (or every time refresh is addressed).
- Semaphores can also be used to keep server resources under control, so that up to X items can happen at the same time, if you know this will be a heavy action.
- RecoverableException, Symfony will automatically try again to process this message if it is thrown from a handler.
- Shared locks: acquireRead can allow multiple users of a lock to read an item but will give an error when multiple updates are made. This way, when updating an order, it can no longer be read until the update is done.
- Cross process: This allows us to create functionality like Netflix, up to 5 sessions can be active on an account at the same time.
Also note to inject the factory into your services instead of the lock service itself. Otherwise every instance of your service has the same lock and your locking is actually useless.
Keynote: Modern Security with Symfony's Shiny new Security Component
The original implementation of the symphony security system is based on that of Java Spring, which is very complex. Its logic is spread over at least 4 different places and there is no centralization of logic.
Symfony Guard came 5 years ago to make it a bit easier - but each underlying component remained the same, only the configuration became a bit easier.
During the conference, the new security component - experimental in 5.2 - was presented.
The first thing Ryan had to say about this was that the user class and access control in controllers/services remained exactly the same, and everything about authorization remained the same.
The changes are all in security.yml and in the auth manager, and also the complete concept of an anonymous user has disappeared from the login flow and security components.
For logged in users there is the concept of passport & badges as provided, a passport is a user and contains one or more badges.
- CsrfBadge with token
Then we work with events to check those badges and perform actions on them. These badges and passports are only available during authentication and do not exist anymore.
The rest of this session is difficult to summarize, it is very thick. The main goal of this customization is to reduce complexity.
This also seems to work, the security system looks simpler than the previous implementation, although my lack of working in a modern Symfony application was a bit annoying here. The last Symfony application I made was already using the guard system - but still I noticed here that this is one of the reasons I am excited about creating a new Symfony application.