We at Netgen have been involved in the eZ community for quite a while now. Our first contribution to Croatian translations of eZ Publish version 3 was almost 10 years ago. Nowadays, we are deeply immersed and engaged in the community. We are open sourcing our work, blogging about our experiences, planning our 5th in a row eZ Publish Summer Camp, and more. Now that the eZ Publish next generation, better known as eZ Platform, was released, we would like to encourage others from the community to follow our example and start adopting the new stack with a quicker pace.
eZ Platform released
A big launch happened at the end of 2015: eZ Platform and eZ Studio were released. It was a long-awaited event which required a huge amount of effort: a complete revamp of a rich featured enterprise CMS product.
I believe many people from the eZ community were excited. Netgen has been a part of this story for years so we were also excited. Two of our senior developers joined this endeavor 4 years ago to help eZ Systems work on the new stack. Around that time, I joined the eZ Innovation Board where we helped the eZ crew get valuable feedback from partners and customers. Netgen started to use the new stack 2 years ago with a hybrid approach and open sourcing parts of what we have implemented along the way. We also rewrote our own base extensions into a new stack bundle over a year ago. Basically, we are adopting the new stuff as fast as possible.
But, the situation is not that simple and there is still a lot to do, both for eZ Systems and the community. Netgen works on a lot of eZ projects, either directly or through partners, and I have recently spoken with numerous people from the community. The general view is that we are still on the transition path. Yes, eZ Platform and eZ Studio were released as the first version which does not need legacy code to work, but this is just one (even though a very big) milestone on the path. I would describe it as an important signal to the eZ community, especially to those still lagging behind with purely legacy projects, that they should finally consider the new stack as the way to go. There are some new stack early adopters like us out there, but many are not on that train yet. The time has come to jump in.
How we got to where we are now
Big changes started around 4 years ago. The important decision on the part of eZ Systems was to start with defining a Public API which will become a strong point in the whole architecture for the future. The work was immense and it was progressing very slowly, but the same applied to other similar projects: Drupal, Magento, you name it. Another great decision was realizing that writing own underlying framework would be just too much (remember Zeta Components). With that in mind, eZ chose Symfony for that purpose and decided to go with the full-stack approach. The eZ community is now benefiting a lot from this decision, although it is not vice versa yet. I guess the adoption of eZ in the Symfony community couldn't have started before the eZ Platform release since it was the first version without any legacy code. We should be able to see how that goes pretty soon.
The eZ Publish 5 version was set to be the hybrid one with both stacks working in parallel. Given the existing projects and enterprise contracts, I would say this was also a good decision. The hybrid version provided us with an opportunity to start learning and working on the new stack. In my opinion, version 5.0 was released too early. I believe many partners and clients expected something more out of that version but the new stack was too weak at the time to handle anything applicable in the real projects.
We started using version 5.2 in the projects, which proved worthwhile. This version had several new features interesting to use, such as better integration with Varnish, deeper usage of the Symfony full-stack capabilities, etc. The problem was that the community was expecting the new administration interface to come along and conclude the migration, replacing the old administration.
We at Netgen were also expecting the new administration, but looking back at it I see that both the community and eZ Systems made a mistake which is still troubling us all: we all underestimated the effort behind building a new interface from scratch. We perceived it as just one of the modules of the whole system that should be implemented based on the user stories and then styled. In fact, the interface is a much bigger endeavor, if not half of the whole revamp. With the release of eZ Platform and eZ Studio we now have a completely new administration UI, though it is still lacking some features the old one has. I have no doubts that eZ will get there, but it is not there yet. This is currently an issue for a significant number of projects as they can't upgrade to a legacy-free code situation. eZ Systems could have communicated better about the complexity of the administration UI part. Maybe they could have taken another route.
Interesting to know, as a side effect of one of our projects (which I will mention later), we did a redesign of the old administration interface and made it hybrid (it has the pagelayout template in Twig and works in legacy mode false). Several of our clients are already using it. I wonder would it have been better if eZ had chosen a similar route for the new administration UI: integrating the old admin interface in several 5.* minor versions instead of making it from scratch. That way, the community would be able to see the progress and use the improvements gradually. Hard to say would that have been better or not, but it would have been more inline with the other eZ Systems’ choices.
Then, some bad luck came. YUI, which is heavily used in the new administration UI, was announced End Of Life. It presents a potential problem for some projects. When customizations are needed in the backend, some people would rather not work with components perceived as deprecated. I suppose it will be replaced in the future in some way, but till then it might be a showstopper in many use cases.
I think that, in general, eZ Systems had more good choices than the bad ones in the last 4 years and that it steered the big refactoring well. I already mentioned that the overall effort was huge and that the work is still not over, so I would like to congratulate them for what they have done so far and encourage them to continue down the path they’ve chosen.
The current situation
At the end of last year, I hosted a hangout with Roland where we talked about the current situation and what is coming next.
In short, eZ Platform with the new administration UI is the open-source layer of the product. It is the baseline for the implementation of projects as well as for eZ Studio. eZ Studio is the commercial part which gives more weight to the Enterprise Subscription that includes maintenance and support for both layers. The version released at the end of last year was Long Term Support release. During the year, we will have Fast Track releases every 2 months. The separation of the free and the paid version is clearer now and that is a good thing.
It is now possible to install eZ Platform (or eZ Studio which includes eZ Platform) and start working on implementing web solutions purely on the new stack. Besides the good features that have been with us for a few years, like Twig templating, Public API, and REST API, there is finally a good support for Solr. It is also possible to manage content via the new administration UI. DemoBundle has been revamped lately to showcase the usage more effectively, but it is recommended not to use it as a baseline for the real projects. Also, the legacy code can be included and used via the legacy bridge. This should prove itself useful for a while.
But, as I already mentioned, the situation is not trivial at all. There are a few things that are or might become troublesome.
- What I see as the biggest problem is the hard choice for eZ Systems between implementing the new features and polishing the ones that are implemented so far. Obviously, there is still a number of features that were heavily used in legacy and are now perceived as missing. For instance: content import/export tools, version management, multiupload, workflow management, faceting, tagging/taxonomy, etc. An average eZ project needs those features, so until those are available it is hard to expect that such project will be implemented or upgraded purely in the new stack. On the other hand, it is also vital that the features that have been implemented so far are put to use, tested in the real scenarios, documented well, blogged about, improved, etc. This is much needed because it would be improving developer experience (DX). Today, thanks to a huge number of alternative products, all possible frictions should be removed so that both newcomers and experienced eZ developers could use it with less hassle.
However, it is very hard to balance between the new features and improving DX, as both are important. - The second problem is that some features, that were part of eZ Publish before, are no longer open source. I am mainly referring to the eZ Flow extension which is just partially supported in the new stack as read-only field type. It was then replaced by a landing page management tool in eZ Studio with an improved UX when managing blocks. On one side, I can really understand the need to put more weight on the enterprise product. It will drive more business to the vendor, which is good. On the other side, many eZ developers are taken aback by this and the explanation is quite simple: the features were there before, now they are not available anymore.
The root cause of the problem is that eZ Systems envisioned the eZ Flow successor to be a marketer tool for assembling the landing pages quickly, while the community used eZ Flow in a number of other use cases, just like we did. eZ Flow was the foundation of the base extensions we use for handling layouts and blocks in all our projects. I am pretty sure there are other eZ Flow use cases in the community used within the community edition. - The third problem are breaking changes that will have a higher or lower impact. For the 16.02 Fast Track Release a breaking change, related to how to store the user credentials, is already introduced. This means that the users can be managed only via the new stack, and user management will no longer work in the old administration via the legacy bridge. The login to the old stack might work, but it may be wise to make a migration script. From an enterprise subscription point of view this is not a problem, because eZ Systems officially supports old administration interface up until 5.4 version. Still, for many projects, either community or enterprise, there is a need to use the old administration. This might be a showstopper for upgrading to 16.02 FTR. And even more breaking changes will come.
Adding new features and maintaining BC is hard. eZ has used the hybrid approach for 4 years to suppress changes that break the legacy stack. It is expected that BC is no longer a priority with eZ Platform. Some projects will have trouble going further with the new stack and the transition. Maybe they will use the eZ Platform's new Solr features on the frontend while still using the old administration for a while, until they are ready to switch to the new administration UI. However, that will not be possible with breaking changes. - The last problem I will mention is not related to any specific feature, it comes from experience. Developing sites on the new stack is a time-consuming process. There are 2 main causes of this. The first is related to the problem of missing features which were developed in legacy (problem no. 1) so they either need to be refactored or fall back to legacy. This will have less impact over time. The other one is related to a different development practice where similar problems need to be developed with Public API in PHP code as opposed to the old practices of using template fetches. Yes, overall it is probably better to put less business logic in template but it is more time-consuming.
Some other problems probably exist as well, I decided to list the 4 biggest ones. They are directly affecting the transition to the new stack. For a new and simple project where eZ Platform and eZ Studio cover all features, there should not be any reason not to try it out. Nevertheless, the existing or complex projects will be impacted by the problems I mention above.
A good place to start is checking the transition checklist and also André's slides.
I would love to give some general advice but the issues are very specific and are depending on the project. Is the installation Enterprise or Community? Is there a frontend redesign in planning that could justify a budget for refactoring? Are there any custom extensions? Are there any customizations of the old admin interface? There are many question that need to be answered in order to make the transition smoother.
So, you think transition is a task way too difficult? With not enough options and too many problems?
Read on, there is hope :)
The importance of the community
Before suggesting the solutions to the above-mentioned problems, let us go back in time once again. Many years ago, I started to do some open sourcing, blogging, and helping out on the community forums. I’d had several years of eZ experience and I’d been driven by the fact that we had been using an open-source product so it was normal to contribute back. After participating in 2 eZ conferences, I started to see how important the community is for the overall success of the product. It was back in 2010 when I wrote the blog post on the importance of the community. Interesting to see it is still applicable :)
A few months after that, I went to the eZ Winter Conference and everybody to whom I spoke agreed with my views. Yes, business goals drive the product, but developers and other contributors are the energy. After the winter conference, I started to worry about the community even more. Therefore, I encouraged my colleagues at Netgen to share more, blog more, and open source more. I even did a conference presentation in Lisbon about the community engagement.
At the same time, we also recognized that we need to bring the community together more often so we could learn from each other and share what we’ve learned and experienced. Thus, the eZ Summer Camp was born in 2012. I took over The eZ Publish Show from Geoff to keep informing the eZ community on the latest news. Last year I noticed that the community is not communicating fast enough, so I kicked off eZ Community Slack which picked up nicely. And I already have plans for some new things coming up this year :)
All the above-mentioned activity confirms what I said in 2010: community is the energy for any open-source product, completely or partially open sourced. It is related to the success of the project. The sheer number of people is not as important as the sharing and engagement levels are.
This is especially true when it comes to big changes, like the ones we have been experiencing in the eZ community since 4 years ago.
Full throttle now!
This blog post is fairly long. However, if you need to remember only one message it conveys, this is the one:
Dear eZ Community, you cannot expect from eZ Systems as a vendor to build everything and open source it for you as well. It is time for the eZ Community to give its best and start contributing back!
Let me explain this. eZ Systems is no longer doing projects. Instead, they are living from enterprise subscriptions. If they are not able to sell the product and make money out of it, the product will stagnate. The eZ community does projects either with an open-source or a commercial license. It is in the best interest of the partners to use the version which is adequate to the use case. If eZ Systems is adding the features too slowly, it is limiting the sales of eZ Studio directly causing both layers, eZ Platform and eZ Studio, to stagnate. I see the solution to this issue very clearly: eZ community should contribute back more, especially now, in order to accelerate the development, the innovation, and the best practices. Being a relatively small community, we have to be aware we should not differentiate between community and enterprise projects we implement, but open source all non specific parts as much as possible.
On the other hand, eZ Systems should do everything necessary to enable the community engagement, to make it as easy as possible and to encourage all the community efforts all the time. I see the Community Board as an important stakeholder in this.
Remember the problems I mentioned above? If there is an interesting feature that you are building for your project, try open sourcing it. It doesn't have to be perfect. Try to tackle some of the features that you are missing or the ones that are missing overall, even those that are on eZ’s roadmap. Talk with the eZ engineering team. Make a proof of concept so that they have a better start and an easier job once they decide to tackle it. Report bugs. Solve bugs. Blog about use cases. Share experience on forums or on Slack. And always encourage your team members to do the same. Only by joining forces can we accelerate things. If we make things only for projects and let the vendor do the features, there will be no acceleration. Everything will continue on as slowly as usual.
The Netgen effect
They best way to teach is to do and to be a good example. This is what we have been doing for years and would like to continue doing in the future. We have open sourced many things related to new stack. Here are the most of them so far:
- eZ Tags and Tags Bundle recently got to version 2.0 with full multi-language support and a full field type support in the new stack. We even made the first customization of the platform UI integration as a proof of concept. It still needs some work, as the complete tags management interface is missing, but we would gladly team up with eZ Systems to finish it and incorporate it in the core. It seems to me that it doesn't make sense for them to do this from scratch.
- eZ Forms Bundle integrates the content creation with Symfony Forms. It has proven to be very useful in a number of projects. Following this, eZ Systems introduced Repository Forms as a more generic solution (it is handling not only content). We kept our bundle since some people might still find it simpler to use.
- Numerous field types and smaller features which we are using (NetgenBirthdayBundle, NetgenEnhancedSelectionBundle, NetgenOpenGraphBundle, NetgenContentTypeListBundle)
- Making a legacy search engine that wraps the new stack search service
- PDF export integration with a service hosted by one of our partners
- Vagrant configuration that we use internally for development. It’s just got some improvements, so expect a blog post about the topic soon.
- our eZ Publish Community Edition maintenance release (tagged as 2014.12) which is already used by several other eZ partners. We would like to move this to the eZ Community GitHub account to declare it as more of a community effort. Hopefully we should discuss this in the Community Board soon.
- A legacy extension for doing the Symfony and new stack coding from legacy if needed
- The integration of eZ Publish 5 with Sylius e-commerce. You can find more details in the blog post we published when we open sourced the integration.
- Shared Summer Camp workshops
It piles up over the years :) Kudos to the whole Netgen team, especially Edi who is our open-source driving force.
We are working on other things as well that are coming in a little while. I already mentioned that our base extension/bundle for doing projects is based on eZ Flow. As we have a lot of community edition projects, we could not build our baseline on the eZ Studio features. There were no options for us but to make what we need by ourselves due to the problem no.2 (some features not being open source anymore). At the same time, we wanted to establish practices on how to implement websites on the new eZ stack in a more efficient way, tackling the problem no. 4 (developing sites on the new stack being a time-consuming process).
These are some of the reasons we decided to invest in creating a layout and block management system for eZ Platform. It is under heavy development at the moment and should be in a presentable state in a few months. Once it is ready and proven, it might help others in the eZ community to be more efficient in implementing the solutions and sites based on eZ or Symfony. We are planning to open source it at least partially, but given the amount of effort we are investing, a commercial part is required as well. It might look similar to the landing page manager in eZ Studio, yet the use cases are relatively different. Our goal is to make a tool for those who implement and maintain the websites, being it developers, designers, or advanced editors. We’ll talk more about it when the time comes.
As a side effect of creating the layout and block management system, we improved the old administration interface. We needed a Symfony-based interface and the eZ Platform UI was not available then. This interface will be useful to all projects that might get stuck on eZ Publish 5 for a while. The main advantage of this interface is twofold: firstly, it gives the possibility of adding the new stack features on top of it; and secondly, it has a modern look and some UX improvement so it should be easier to use. We will not open source it, but it will be available under commercial license. A demo will be ready really soon!
Also, Summer Camp 2016 with eZ, Symfony, PHP, and other topics has already been announced. More information is coming your way this month.
Conclusion
Congratulations on reading so far :)
It proves you are interested. So, instead of giving a conclusion, I am asking you to think about what you as an eZ community member can do to contribute back. There must be something. If you are not sure, ask for an advice or help.
Actually, just do it :)
P. S. This post was in the plans for a few months now and yet it somehow unintentionally coincided with me being chosen for the Community Board last week. I hope this will give me a chance to promote everything what I wrote in this post even more.