It’s been a week since the application phase ended, and now it’s time to announce the results.
First of all, thanks to everyone who submitted an application. There were even more applications submitted than last time, and after discounting duplicates, the total number is 616 applications, including everyone who applied last time. You keep outperforming my expectations 😛 I’m glad that the community shows no signs of dissolving or diminished interest even though it’s now 7 years since the game closed. I’m confident the day will come when everyone will be able to play the full game again, and I’m glad to know the community will still be around.
This opening has also shown some weaknesses in the current application processing infrastructure. There was no easy way to double-check whether your application was still recorded if you had submitted one last time, and also no easy way to check that your application was submitted properly if you submitted it in this opening. The integration with website accounts, discord, and the LU server itself also isn’t ideal at the moment. For future applications, I’d like to develop an interface where you can submit an application and check on its status later on, maybe even as an interface of the LU server itself.
Because of this uncertainty in whether your application had been processed, some users submitted additional applications under the same name. Of course, these won’t disqualify you from the alpha, and we’ve also taken steps to ensure everyone is only counted once.
I’ve taken the time and read through every application that was submitted, and they’ve been a great read, with many of you putting a lot of effort into detailing your memories of this game. Thank you all for sharing these nostalgic moments.
There were a lot of wonderful applications, and it’s sad we can’t select all. I really hope we’ll be able to do another opening soon, so that you’ll have another chance at getting to play the game. However, today, let’s take a look at the selections for the current alpha opening.
We’re randomly selecting 20 players out of 616 applications, which means if you entered you have a 3.24% chance of being selected.
The selected player for the alpha opening are:
Congratulations to all of the above. The selections are quite varied, some applied this opening, some applied last time; some were FTP players, some were members, and some never got the chance to play.
If you’re one of the selected, check your mail, info on how to connect has been sent to your mail address. (Check your spam folder too, just in case.) If you’re not one of the selected you may still have a chance in later selections, there will be a number of openings in the future.
I’m glad to welcome 20 new testers to the server. Every opening brings us one step closer to having the full game working again.
With the new network layer working, the time has finally come for another opening in the current alpha. It’s been one and a half years now since the start of the alpha, and a lot of things have changed since then:
The alpha provided some useful real-world experience of how the server responds to a number of players being connected at the same time. I initially only scheduled the server to run on weekends, but as the server ran better than expected, I switched the server to run 24/7.
The alpha testers have found and reported more than 100 issues, and a lot of them have been fixed already. There are still a bunch of issues remaining, but with the network layer rewrite functionally complete and other rewrites planned, the project is moving closer to fixing them.
The work on the MLN server has helped to both get a revival of a related game going, and to get some insight in how to design the rewrites needed to address the most problematic bugs found during the alpha.
The first of the rewrites has now completed the minimum viable implementation, and work is under way to make the new network layer work entirely under the hood.
The rewrite of the network layer has at the same time removed a large blocker for a new alpha opening, and introduced a new component that would benefit a lot from real-world testing. Testers have previously experienced connection issues that would result in their connection getting dropped, especially during world loads. The network layer rewrite attempts to fix that, but connection issues are notorious for only happening to some specific users in specific situations, so having more testers would help in making sure the issue is resolved.
A new opening in the alpha is also a good opportunity to keep testing active, as the activity naturally slows down a bit over time.
With these factors coming together, it’s time for a new opening up the alpha to new testers. The server development is still very much focused on bug fixing and getting the parts that are implemented to work like in the original. This is my top priority because there’s not much point to features if they don’t work right. That’s why this new stage doesn’t mark the introduction of new features to the server. However, there has been progress in server development since the initial alpha release, which is why we’ve decided to launch a new stage. To avoid misunderstandings, we’ve chosen to call it “opening in the current alpha”, as opposed to “new alpha”, and we’ll be using the latter only for releases with new features.
With all that out of the way, it’s time for the specifics of the new opening. We’ve chosen to keep much of the process we used for the initial alpha, as it’s worked quite well.
The new alpha opening will start on April 26th. On this day the new testers will be able to join the server and play.
There will be 20 tester slots available for this opening. Twenty new people will double the count of testers with access to the server.
People wishing to be testers will need to apply to the alpha with a short description of why they want to be a tester. This system will help to ensure that the selected are dedicated, and will discourage duplicate/spam submissions.
If you had applied to the initial alpha, you will be counted automatically, without having to resubmit an application. Don’t worry, we haven’t forgotten about you.
Those eligible to be selected (having written a valid application for any alpha opening) will be selected randomly by a computer. This will ensure that everyone has equal chances of getting in.
The period in which you can submit an application starts now, and ends two weeks from now, on April 9th.
We’ll be looking through the applications the week after that, and announce the results on April 16th. The selected testers will receive a mail with instructions and will be able to access the alpha tester channel on the project discord.
Finally, on April 26th, the selected testers will be able to join the server and play.
We’ve written up some more detailed explanations in our Alpha FAQ. If you want to apply for this alpha opening, click the link below.
It’s been one and a half years now since the initial alpha opening, and I’m glad we’re now finally at the point where we can let more people play. The server still has ways to go until it will be complete, but it’s slowly and steadily progressing, and I’m confident we’ll get there one day. With the new alpha opening, we’re a step further along.
When I was working on the MLN server, one thing that made development much easier is that the lowest levels were automatically taken care of, as much of the work had already been done by the TCP and HTTP protocols. In contrast, my work on the LU server has led to me having to start at the bottom and implement the RakNet protocol, which takes the role of TCP in LU’s networking. My experience with the MLN server got me thinking that perhaps it was possible to redesign LU’s networking so that the benefits of MLN’s networking could also be applied to LU. This is what led me to develop a new network layer that I’ve now finished work on.
The protocols involved here are situated at the transport layer of networking. At this layer, there are two protocols that are widely in use: UDP and TCP. These protocols are involved in basically every traffic on the internet. UDP is an extremely simple protocol, and essentially all it does is providing ports to associate network packets with a specific process on a computer. At this layer, network traffic is still very unreliable; packets sent from one computer may get lost in the network and never arrive at the destination. They may also arrive in the wrong order, which is also important for many applications. UDP does not attempt to solve these problems, and therefore packets sent using the UDP protocol are unreliable. TCP however implements special mechanisms under the hood to ensure that packets will reliably arrive, and will arrive in the proper order. A lot of applications (websites, email, etc) need packets to be reliable, but there are some for which unreliable packets are actually useful, like DNS. However, for these applications, this is usually a clear choice, they’ll either go full unreliable or full reliable.
Games however pose a special case in that they sometimes need reliable packets, and sometimes need unreliable ones. Here are some examples from LU:
Logging in to the game needs to be a reliable packet, because it’s important that the server receives the login request – if not, you’d be stuck at the login screen forever.
The position updates sent by the client when the player walks around don’t actually need to reach the server 100% of the time – they are sent so frequently that it doesn’t matter if one is lost. Therefore position updates are more suited to unreliable packets.
Therefore LU can’t just decide on one of UDP or TCP and stick with it, it needs features from both. This is what the RakNet protocol tries to solve: it allows LU to choose between unreliable and reliable transmission for every packet. RakNet works by using UDP as a base, and then builds mechanisms on top that ensure reliability. But unlike TCP, these mechanisms are optional, and can be enabled per packet.
This seems like a solid solution for this problem. However, RakNet has a few quirks in its implementation that make using it less than optimal:
In addition to the distinction between “reliable” and “unreliable”, RakNet also supports some more fine grained options that are somewhere between raw UDP and a full TCP implementation. However, at least in LU’s case, these special modes are almost never used – it’s usually all or nothing.
RakNet has some flaws in its protocol design that make it unnecessarily complicated, the same could be achieved more elegantly.
RakNet is a niche protocol, with far less support than UDP or TCP. This makes it more likely to have bugs and inefficient implementations.
LU’s copy of the RakNet implementation is statically compiled into the client .exe, which means it can’t be updated, especially since LU itself is no longer updated.
RakNet implements its own combination of cryptography to encrypt its connection, and the encryption isn’t enforced. Together with the fact that RakNet is niche and more than 10 years old at this point, the protocol cannot be said to be secure.
Thus, there’s room for improvement for LU’s networking situation. While working on the MLN server, I had an idea for a possible solution: The same kind of per-packet distinction could be achieved by using both UDP and TCP simultaneously, sending on UDP when unreliable packets were needed, and on TCP when reliable packets were needed. The solution seemed both simple and obvious enough that it was more a surprise that LU wasn’t using it already.
However, even though this seems straightforward in theory, LU’s situation made it quite complicated to actually implement in practice. We can’t just switch LU from one protocol to another, since RakNet is baked into the program and can’t be removed. Therefore, what I’ve been working on the past few months is a “shim”: A program running on the client side, acting as a local server for the RakNet protocol, translating the traffic to the TCP/UDP based protocol, and relaying it to the actual server. Due to the way LU works, this has been a bit more complicated than initially thought, as the shim also needs to simulate multiple server instances for when the player switches worlds. However, after a lot of development and testing, I’ve been able to get it fully working.
With the TCP/UDP protocol fully working, it was straightforward to swap out the TCP protocol for the encrypted TLS protocol, which is widely used, for example to secure HTTPS connections. With the protocol now using this cryptographic industry standard, LU’s security is now rock solid.
Additionally, the UDP and TCP implementations are provided by the operating system, so they can be improved without the program having to change. This also means that they are optimized to the full extent possible, as the same implementations are used for things like high speed file transfer and video streaming.
I’ve done a few unscientific benchmarks, loading worlds in LU with the old RakNet protocol and the new TCP/UDP protocol, and there are some quite noticeable improvements:
Values are time from when the client signals clientside load completion to the end of the loading screen, on localhost, without encryption, in seconds:
Edit: It seems my connection or my ISP’s firewall may have significantly slowed down RakNet’s loading times in this test. It appears RakNet can be significantly faster on better connections, but the new protocol still seems to outperform it there.
Previously, loading a world could be quite slow, mostly because the congestion control and retransmission timeout algorithms used in my RakNet implementation seemed to have some problems estimating the ideal values for the amount of packets per second to send, and how long to wait until the packets would be resent. With the new implementation, everything is lightning fast, and the complexity is outsourced to the TCP implementation of the operating system.
However, it’s possible to make these improvements even better. To keep backwards compatibility with the RakNet protocol, right now the shim needs to run in the background all the time as a visible console application. The client’s boot.cfg also needs to be changed to to configure the client to connect to the shim instead of connecting to the server directly. This isn’t as user-friendly as I’d like it to be. The complexity of running the shim and translating between the protocols also adds some overhead, though it’s usually small compared to network latency.
However, these things can be fixed. As I’ve mentioned above, the RakNet implementation is baked into the client .exe – it’s not possible to replace it, as you’d need to be able to recompile the client. This isn’t 100% true – there is a way of modifying the client to run code instead of RakNet’s network functions: .dll hooking. Dll hooking works by getting the client to load a dynamically linked library at startup. The code in this library then has access to the internals of the client process, and with some surgical precision, it can find the location of the machine instructions of the RakNet functions and hijack them to run its own code instead. This technique has been successfully done in a number of games to mod them without source code. However it’s also much more intricate and complicated than the shim solution, which is why I chose to focus on completing the shim first to show that this kind of protocol replacement is possible and valuable. Now that that’s done, I can further look into a .dll-hooking-based solution, which should in theory be able to provide record speeds at less overhead.
In the meantime, I’ll be releasing both the shim executable and source code in the next days, so that other projects are also able to make use of this new protocol.
The network layer has been very important to me in making a decision about opening the ongoing Alpha test to new players. Some alpha testers have previously reported getting randomly disconnected from the server sometimes, which this new protocol should fix. Along with the faster loading times and the increased security, this new protocol is vital in improving the user experience of the testers. Therefore it has been an absolute requirement to me for a new alpha opening. With the protocol completed, the outlook for an alpha opening now looks much better… but more on that in the next progress report 😉
As I’ve mentioned on twitter, I’ve been working on a side project for a little bit, which like my main project is a server for an abandoned LEGO game. I’m now ready to announce what I’ve been working on: A server for My LEGO Network.
My LEGO Network was a mix between a social network and a game, made in Flash and accessed via web browser. It was around from 2008 to 2015, when it was shut down, like many other LEGO games before it. Some of you may remember playing it back when it was online.
I played a bit of MLN back when it was online, and although the game seemed quite grind-y, I liked the general concept of interacting with both players and networkers and expressing yourself on your public page. The networkers were interesting and varied, and they were connected to other networkers themselves, which made the concept of a LEGO social network more immersive as a game element.
A few months ago, Knightoffaith, a fellow LU server developer, mentioned he was looking into recreating a server for MLN. Having played the game years back, I was kind of curious how it worked behind the scenes, so I started investigating. Knightoffaith had already gathered (almost) all client and resource files, which made it possible to assemble the client side without too many problems. (Thanks to Knightoffaith for this, along with organizing and leading the project. Without him the restoration effort probably wouldn’t exist, or at least wouldn’t be very far along.) Using these files as a starting point I was able to research the network communication and put together a server.
I’ve managed to implement most of the core features of MLN by now. Here’s a list of what’s working so far:
The Private View, including setting your avatar, setting your profile statements (“about me” bio) and managing your friends.
Reading and sending mail, including support for attachments and the “easy reply” feature (suggested replies).
The inventory and blueprints are also working.
There’s also support for MLN’s integration with galleries, Factory (LEGO model upload), and LU’s creation lab (works similar to a gallery).
Modules can be placed on your page in the page builder. There are several different ways of module customization or configuration, of which all are supported.
Public pages display your badges and modules. You can also send friend requests to page owners.
You can vote on modules, set up modules with items, and harvest items that the modules have grown.
Arcade modules are playable as well.
Unfortunately, while a lot of resources and clientside database contents had been archived, there is one key piece that is missing: Data about networkers. Since networkers were a server-side feature, they don’t appear in the client database that was archived, and since MLN used flash, their pages weren’t archived in automated web crawlers. This means that we don’t have data about networkers’ names, their pages, their networker friends, the requirements to be their friend, and their mail responses.
Of course, the ideal solution would be that through some sort of miracle someone stumbles upon the networker database somewhere. However, even if this doesn’t happen, it should still be possible to recreate the database – but it will require some manual work. The MLN wiki has a lot of info regarding networkers archived, including page and mail trades. Mail responses can also be guessed from the mail contents (which, due to the way MLN’s message system works, are archived in the client database), as networkers usually mention what they’re sending and why they’re sending it. However, this leaves the page layout and the networker description & quote on their page missing. The only way to recover them would be if someone happened to record the pages or take a screenshot, back when they were online. Hopefully we’ll be able to piece these things together at some point.
By now you’ve probably got some questions about this project and my involvement with it. I’ll try to answer some of them below.
Why are you working on another game server? Shouldn’t you stick with LU and focus on development there?
It’s true that working on one server takes time that could be used for the other. I knew this when I starting getting involved with MLN, and I considered both continuing looking into a server for MLN and sticking with the LU server. In the end, I found there were a few reasons that made exploring an MLN server worthwhile:
Exploring different database options:
The LU server has had some bugs caused by the database system it uses, which unfortunately are very hard to track down and fix. I’ve been curious for a while now how other database systems resolve these kinds of issues, and whether it would make sense to switch the current system. With the MLN server I was able to use the Django framework and its database ORM, which have been widely used and tested. They also work a bit differently than the LU server’s database system, which prevents some kinds of bugs I had previously encountered.
Exploring web interfaces & Django:
I’ve got a bunch of ideas for administration and overview interfaces for the LU server that would ideally be implemented as a web interface. However, I didn’t have a lot of experience with modern data-driven web development, and working on a server for MLN seemed like a great way to get some in-depth experience that would help with the LU server later on.
Exploring open-sourcing a working game server:
I’ve always wanted to make my LU server open source at some point, to make more servers available to players and to decentralize the server landscape. However I haven’t been able to do so so far, for multiple reasons, including the LU projects situation and to avoid LEGO taking legal means against server projects while the server still requires a lot of development.
The MLN server project offered an opportunity for an open, central, community project working to recreate a game, instead of the current LU situation with multiple closed projects. The MLN server makes it possible to gauge the impact of an open source project on the community, which will hopefully come in useful when the time comes to decide whether to make the LU server open source as well.
Exploring working in a larger, community-based effort:
For similar reasons as with not being able to open source the LU server, I haven’t been able to include more people in my LU server project, the situation is quite complicated. With MLN, it’s possible to “start things off the right way”, with a single, open project that makes it easy to collaborate to bring the game back.
MLN’s restoration is also inherently more community-based than LU’s: Writing a server for LU mostly requires research and development of LU’s game mechanics, which are tasks that can be done by a relatively small team of developers. In MLN’s case, research and development takes a much smaller part of the restoration effort, and the remaining part of piecing together networker info is much more suited to a crowdsourced approach. I’m looking forward to seeing the restoration effort working with MLN fans to recover networker info.
Why haven’t you talked about this earlier?
Investigating the possibility of an MLN server and developing a prototype were some ventures that were quite experimental, and I first wanted to make sure that a working server for MLN was actually feasible before announcing my involvement in this project. I’m also not a big fan of releasing unfinished things, and wanted to make sure almost all core features were supported before a release.
Will you change your focus from developing the LU server to the MLN server from now on?
Actually, no. Part of why I chose to write a server for MLN is that MLN is far less complex than LU. By now most features are implemented, and there’s a strong foundation for other developers to pick up on my work and implement additional features. From now on, my work on MLN will consist mostly of occasional feature updates and bug fixes. Most of my time will be spent on the LU server, like before.
Can I actually play MLN again?
Sort of. The core system is implemented, however, without networkers, there is no story progression and no rising in the ranks, and you’re limited to the social part of MLN, customizing your page and interacting with your friends.
Can I try it out?
There’s no public hosted MLN server, and I don’t plan on hosting one myself, as I’d like to focus on development and less on server operation. However, the server is open source, so if you’re interested, you can host your own instance to play on. The source is available here, and contains instructions on how to install and set up the server. However, to avoid any copyright problems, I can’t link content files.
What’s next for the MLN project?
There are a few details I haven’t gotten around to implementing yet, like statistics and modules sending out items to random friends. These should be implemented in the future. Other than that, the main challenge is to assemble comprehensive networker data and build the necessary server infrastructure to support networkers.
What’s next for you?
I will be returning to focus on developing the LU server, making use of what I’ve learned from the MLN server where possible. However, unspectacular work like bug fixing and test writing will still be a sizeable part of my work. These things are however necessary for effective development of a large project like this, and will improve the server quality and prevent regressions in the future.
I can’t yet say when I’ll be able to return to implementing features, but I’ll let you know when the time comes. Once I’ve done enough work on features I’ll also go back to recording videos for the youtube channel. Unfortunately bugfixes don’t make for very interesting videos, which is why I haven’t uploaded videos recently. Once I’m back to developing features videos will be possible again.
I also can’t yet say when the next alpha will happen. I know you can’t wait to play LU again, and I want to start the next alpha soon myself. However it’s very important to me to make sure the server is ready to handle more players before inviting more players to the server, and to fix as many bugs as possible so you can have a fun experience and play like in the original LU.
This is why I’ve put so much focus on bugfixing and testing in the last months. I don’t want to rush the next alpha and invite people while the server still has a few bugs around. Operating the server is only going to get more complicated with more players, which is why I want to fix bugs as early in development as possible. I’m sorry development is taking so long, but I hope you can understand why.
Can I help with restoring MLN?
Yes, this is fortunately much easier than helping restoring LU. The server is open source, so if you have experience with Python and Django, you can help with software development. MLN doesn’t really have special formats you’d need to learn, so getting started is relatively simple. However, keep in mind that this requires programming experience, so this way of contributing might not be for everyone.
However, you can still contribute, even if you can’t program. As mentioned, we don’t have a lot of data about networkers right now, and if you played MLN in the original days and remember some details that aren’t on the MLN wiki, or even better, have screenshots or videos of networkers’ pages, it’d help a lot if you could send us info or update the MLN wiki.
However the restoration has only just started, and there isn’t much organization yet regarding the collection of networker info, so please keep that in mind if you are interested in helping out.
In conclusion, I’d like to say that working on the MLN server has been a lot of fun, and I’ve learned a lot working on it. The server gives the MLN restoration project a solid base to work with, and I’m confident they will be able to restore MLN to its original state in time. The work on the server has both given me new ideas for the LU server as well as means to achieve old ones, and for example something like a web interface for the server for both admins and players doesn’t seem too far-fetched any more.
So, by now it’s been three months since the alpha started. Time for some updates on what’s happened in the last three months.
You’ve probably noticed that there haven’t been too many updates on progress lately. That’s partially because I’ve been pretty busy and haven’t had as much time in the last months, but also simply because alpha progress isn’t as easy to demonstrate as feature implementation progress. There’s been a bunch of improvements since the alpha started, but most of them have been minor or internal, and not the typical stuff we can show in our videos. However, by now there are enough that I can talk about them in more detail.
So here we go, a comprehensive list of bugs that were resolved and features that were implemented since the start of the alpha:
(Issue titles usually appear as the tester submitted them)
Pyraknet: Issue: Duplicate detection not working properly.
Pyraknet is the network library implementing the RakNet 3.25 protocol that provides the low-level network layer that LU uses for network communication.
It needs to resend packets when the client doesn’t receive them for some reason, and it needs to be able to detect and throw away resends it doesn’t need when it did receive the packet.
In a rare case (only 1 message used by LU was using this packet type) the duplicate detection didn’t work. This patch fixed this problem.
Issue: Block Yard doesn’t work.
Most objects on Block Yard didn’t appear for some reason. It turned out this was due to a change in the server’s spawning logic, which, ironically, actually was implementing a new feature, but invalidated an assumption made by Block Yard’s world script. The issue was quickly fixed with some changes to the script.
Implemented: Achievements for collecting powerups.
A rare achievement type, used only by 3 achievements in the whole game, was implemented after a tester noticed the achievements not working properly.
Issue: Rocket Take-off uses another rocket than the one placed.
Selecting a rocket using the mouse rather than pressing shift wasn’t working correctly, and got fixed.
Issue: Imaginite isn’t taken by mission.
The mission in Nimbus station where you can trade imaginite for backpack space wasn’t removing the imaginite properly. Fixed.
Issue: Turret quickbuilds don’t despawn.
Avant Gardens’ robots drop turret quickbuilds, which only despawned when built by a player. Most of the time the players didn’t build the turrets and only smashed the robots, leading to quickbuilds piling up on the AG battlefields. Fixed by introducing a self-destruct timer.
Issue: Imaginite isn’t taken by interactables.
Similarly, survival and wishing wells also didn’t remove imaginite on interaction. This also got fixed.
Issue: Survival enemies don’t spawn.
Similar to the Block Yard issue, spawning logic was also affecting AG Survival. Similarly fixed with alterations to the script.
Issue: Unimplemented achievement rewards.
A few achievements didn’t give out rewards when completed. It turned out this was due to some entries in the game’s original database not being set correctly.
Implemented: Achievements for survival times.
Most achievements regarding survival times were already implemented, but it turned out the game actually uses two separate mechanics for these achievements, and only one of them was implemented before.
Issue: Monument finish line won’t end quickbuilds.
A quickbuild wasn’t completing correctly. It turned out this was because the object overrode the game’s database on quickbuild complete times. Fixed by implementing the override.
Issue: Imagination fountain doesn’t drop imagination.
Some items that were supposed to drop powerups, didn’t. Fixed by implementing the associated script.
Pyraknet: Issue: Absolute time used when relative time needed.
In some packets pyraknet has to send timing information in packets. It turned out the time it was sending was absolute on Linux (but not on Windows), when it should have been sending time relative to program start. This change enforced using relative time everywhere.
Implemented: Random missions.
The game features some daily missions that are drawn from a random pool and not offered sequentially. Support for this was implemented.
Issue: “Stagecraft” achievement not working.
Another bug caused by a script not working correctly. While some more work on the script remains to be done, it now correctly updates the achievement.
Issue: Mission to play a guitar not working.
Similarly caused by a script, and similarly had some complications appearing that caused full implementation to be postponed. The mission is correctly updated now, however.
Issue: Paradox’s Plasma Ball 1 does not kill enemies.
Caused by a bug in skill parsing. Skill structures remain relatively unknown, mostly due to complex conditions that are hard to reverse engineer. A few similar bugs for other items still exist, and in some cases investigations are pretty much stuck because the condition can’t be analyzed even when the matching code is found. In this case however it was possible to resolve the bug.
Issue: Plasma Ball 1 jump attack does nothing but consume imagination.
In some instances, skills can cancel their execution, as happened with the item here, which doesn’t cast its effect if the player is jumping. The server was however still removing the skill’s imagination cost, even if the skill cast had failed. This was fixed to only take the cost if the skill actually succeeded.
Issue: Spider Cutscene being played for everyone
Some network packets are supposed to be sent to all players, while some should be sent to just one player. In this case, it seems a message that would normally be sent to all had its behavior overridden to only be sent to one player. The server didn’t know about the override, and sent it to everyone, causing the cutscene to appear for everyone in the world. Fixed via override.
Issue: Rocco Sirocco doesn’t spin
A script wasn’t working properly, and wasn’t displaying a cinematic when the mission completed. Fixed through script modifications.
Issue: Daily Mission “Six Shooter” doesn’t give Plunger Gun.
A script wasn’t implemented, and didn’t give out a mission item.
December featured several large refactors, which reorganized the server internals and cleaned up the code. They didn’t change the external behavior of the server by much, but nonetheless represent significant improvements, as they affected about half of the server and made it easier to implement new features in the future.
Refactor: Server command system.
The server command system was rewritten to be completely separate from the core server. As a result of this change it’s now possible to delete the entire folder related to commands, and the server will continue to function as normal (just without commands, which aren’t part of normal gameplay).
Implemented: Mail attachments.
Mailing attachments had been supported for a while, and the server was already using it to distribute achievement rewards when inventory space was full. However the interface allowing players to send items per mail was disabled, until this patch changed this.
Implemented: Mail cost.
Sending mail costs coins, and mailing achievements increases this cost by the cost of the items mailed. With this change, the server now subtracts the correct cost from the player when sending mail.
Pyraknet & LU server: Refactor: Separate BitStream into read-only and write-only versions.
Pyraknet also provides a way of reading and writing binary data sequentially, and which allows reading/writing single bits. It turned this was almost never used for reading and writing from/to the same stream, and therefore could be separated to make the intent of the stream more clear.
Pyraknet & LU server: Refactor: Change to use composition instead of inheritance.
Pyraknet had previously been using an inheritance model, where the LU server overrode certain things to implement its own functionality. This worked well and allowed for a lot of freedom for overrides, but didn’t separate the low-level pyraknet stuff from the high-level LU server stuff well. This was changed to use composition instead, where the LU server interacts with the low-level layer mainly via callbacks and delegations, which provided proper separation.
Pyraknet & LU server: Refactor: Add type annotations.
Python is dynamically typed, and as such doesn’t enforce a variable to remain the same data type. However information about the intended data type is useful for documentation and static validation, and so Python has added support for this information in recent versions. With type annotation support improving, it was time to add type information to the pyraknet & LU server projects where it hadn’t already been present.
Pyraknet & LU server: Refactor: Various refactors.
A bunch of minor things were changed and improved, some variables were made more restricted, and some internal systems were reorganized. The changes are relatively specific to the internal implementation, so I’ll keep this paragraph short. However, that doesn’t mean these changes were insignificant – grouped together, they were one of the larger refactors.
Issue: Buttons at the AG Monument trigger even when not built.
Buttons could be pushed even though they hadn’t been built yet, also known as “invisible buttons”. This was caused by an unimplemented check in the component, and was fixed by implementing the check.
Pyraknet & LU server: Refactor: Restrict bitstream interface further.
Previously it was possible to access packet data both by array index notation and via the sequential interface. In practice the random access was rarely used, so this patch removed this functionality and restricted the interface in some other points as well, which makes passing the bitstream to other handlers safer.
I hope this list could give you some insight to the kind of bugs that pop up in alpha testing, and to what I’ve been working on in the past months.
There have been a bunch of bugs fixed since the start of the alpha, but there’s plenty left to fix. 50 bugs remain open, and while most of them should be possible to fix, it will take some time. I want to make sure the server doesn’t just have a lot of worlds, but also runs smoothly, and for that to work, it’s necessary to fix the open bugs before opening the next alpha stage. Based on the time it took to fix the above bugs, it’s likely it will take some months until the server is ready for the next alpha. But by then, the server should be pretty robust. 🙂 I’ll publish something when the server’s ready, and I’ll continue the progress updates in the meantime.
It’s been a week since the application phase ended, and now it’s finally time to announce the results.
First of all, thanks to everyone who submitted an application. In total, we received 303 applications, more than I’d ever have expected. I’m really impressed with how many people are still around who want to play LU, even though the game closed 5 years ago, and how many would apply even though this is just a first alpha. I went ahead and read every one of those 303 applications you sent us, which took a couple of hours, but I felt it was the least I could do to honor your dedication to this game. Your applications were much more than I imagined, with many of you detailing fun moments in original LU, your enthusiasm for the game, and your excitement for being able to play again. It was a great and inspiring read, thanks to you all for taking the time to write all this.
The application system also seems to have worked well to prevent spam; out of 303 applications only 1 was spam (and don’t worry, we didn’t accidentally classify your application as spam, the application in question was a clear impersonation we manually confirmed before disqualifying it).
Now, before continuing with the selections, a note on how the system works: You submitted your application in the application phase. From these 302 applications we selected 20 to be included in the first alpha stage starting on October 26th. The selections were made randomly, using a computer (to be precise, a small python script). However, these 20 people are just the first stage. To evaluate the server performance under load, we’re starting with a small amount, and will add more people in later stages. This also means that if you submitted an application and didn’t get selected, you still have a chance, because your application also counts for the later stages. If you haven’t had a chance to submit your application, you will also still have a chance in later stages, where we will be accepting new applications as well.
Now, on to the selections!
We’re selecting 20 players out of 302 applications, which means if you entered you have a 6.62% chance of being selected.
The selected players for Stage 1, starting on October 26th, are:
Congratulations to all of the above. I’m glad the random algorithm provides for so much diversity. We’ve got all kinds of LU fans in there, from original beta testers to live players, to FTP players, to people who never got the chance to play, people from the developer side and people who are engaged in the community, young ones and AFOLs who played LU with their kids. I’m really looking forward to all of them coming together in-game to play LU again.
If you’re one of the selected, check your mail, we will provide you with info on how to connect later today.
If you’re not selected you may still have a chance in later selections, as mentioned above.
And no matter whether you’re selected or not, we will continue to share info about our progress and the alpha with you. You’ll be able to look forward to alpha gameplay videos not only from our youtube, but from the players as well, as we allow streaming and recording of the alpha.
Remember, this all starts on October 26th! I’ve been looking forward to this next step towards everyone being able to play again for years now, and I’m excited that it’s only 5 more days until then.
We’ve done a lot of server testing over the last month. I’m glad to say that we’ve been able to fix the bugs that came up, and implement the necessary server administration features. We believe we’re ready for a first alpha, where we will invite people from the community to test our server.
The first alpha stage will begin on October 26th, 2017.
We’ve mentioned before that we’d like the alpha selection process to work in a fair way, and we’ve come up with a process that we think will help with this.
We’ve decided not to base alpha access on giving out keys, but instead via applications. Everyone who wants to join the alpha can submit an application describing why they want to join, and after filtering out spam we randomly select from the submissions. This will help with duplicates, and complications with people asking for keys.
We’ve decided to start low with 20 slots available for the first alpha stage. This is so we can analyze how well the server responds to multiple players playing simultaneously, and possibly optimize the server to work more efficiently under these conditions. After we complete our analysis, and implement bug fixes and optimizations, we will expand the alpha in later stages to include more players and features.
To explain the process, here’s a timeline:
September 30 (Today)
Alpha application phase starts. From now on, people are able to submit their application for alpha.
Alpha application phase ends. People will still be able to submit applications, but they will count towards a later stage, not the one beginning on October 26.
After filtering out spam and randomly selecting applications, we announce the selections on October 21. Selected applicants will be notified via mail about the next steps.
Alpha Stage 1 starts, with 20 selected people.
Once testing with the initial group of 20 people is successful, we will open up the alpha to more people in later stages.
We’ve written up some more detailed explanations in our Alpha FAQ. Check it out if you’ve got a question, or post in the forums.
You’ll be able to submit your applications to the first stage starting now, until October 14. Click below to start your application for a chance to be selected.
This is just going to be a pretty technical short summary about what we’ve been doing this month. Don’t expect any big announcements here, think of it more as a “behind the scenes” look at what we’ve been working on. Server development is just one of multiple things we do, so there’s more than that listed here.
Note: Before I start, I’d like to mention something:
In the last post I mentioned that for server costs, we’re going to have to set up some sort of donation system at some point. I also mentioned that we’d like to thank donators in some way, and suggested an in-game cosmetic shirt for this. Some of you have responded that you see this as too similar to “freemium” or “pay-to-win”. We only planned this as a small thank-you, with nothing more in mind, similar to something like donator mentions on twitch. However, we understand your concern, and I want to assure you that we will never be “freemium”, everything is and will be 100% free, no catches. Therefore, to leave no questions open, we have decided not to give out in-game items at all. Rather, items will only be available through gameplay. We still want to thank everyone who considers donating to us to help us with the server, but we want to make clear that the playing experience is the most important thing to us.
Now, on to the actual progress report!
Investigations on older lvl formats. Lvl files are used by the game to store world information like NPCs and Enemies, and are vital for a server to work.
Investigations on the game’s audio formats.
As a side effect of the above investigations, it turned out the current structure definition language used to document formats is not completely capable of describing all formats. Currently missing are
Support for pointers in formats
Support for recursion
Support for definition of larger structures from smaller ones
This needs to be added at some point to be able to describe more complex formats. The struct definition language is used by the captureviewer and structparser tools to parse binary files and packets. Luckily so far it has sufficed for them, which is why these issues have only surfaced now.
Work on a struct-visualizing hex-editor to help with internal research on formats. Screenshot:
It’s still experimental and too early to release, but it’s helpful for investigations on file formats.
Work on the client’s fdb database format.
Work on cataloging available clients. For research purposes we are always interested in clients from older versions, especially from beta and alpha.
The docs are now also available in a faster-loading read-only mode.
I’d like to do some more documentation maintenance at some point, but it’s difficult to find time.
Work on improving contribution-friendliness to the docs, in cooperation with researchers from other projects. We always welcome contributions, especially if they are well-researched and -sourced.
Pyraknet is a minimal port of the network library used by LU, RakNet 3.25, to python. It’s open source and used by multiple projects.
Work on making the library more robust, featuring refactors, tests and type annotations.
Implemented support for split packet receiving. This was not possible before due to having no proper way of getting the client to send large amounts of data.
Investigations on maximum packet size. It seems LU has a packet size of 1200 bytes hardcoded for some reason, even though it’s not really necessary in the network. But just to be sure pyraknet will also use this lower size from now on.
Features seen in our video for this month. Side note: LU is bananas.
Ongoing work to test the server against edgecases, high loads, network issues and others. This will take a lot of our time, so it’s pretty likely we won’t be able to release a video next month.
The website has been online for a month now, and everything’s working well. Thank you for your threads in the forums, I’m glad the atmosphere is so nice there.
And of course thanks to everyone commenting on twitter and youtube as well. I read all of your comments, and I really appreciate them 😊
Notes on Alpha
It will still take a few months to prepare everything for an alpha release. Please be patient until then. We’ll post updates about release and admission phase dates once we have something to report, so you don’t have to worry about missing them.
We’ve finally gotten around to launching a website. So far we’ve focused on server development, but now we’re ready for a public-facing display. There’s project info, FAQ, media and ingame screenshots, and what I’m looking towards most, forums, which brings our project one step closer to being a complete community. And of course there’s also this dev blog, where we’ll be posting longer updates & announcements, as well as some write-ups about the internals of LU. (For shorter updates check our twitter as well).
Got suggestions? Something not working? Leave a comment or post a topic in the forum.
Plans for the future
The launch of this website clears another checkmark off our list towards a first alpha release. We believe the server is mature enough at this point, and have made sure that the features are working well. We’re working hard to deliver to you the server you deserve, and the first step in that direction is an alpha test where we will host a small server instance to a limited set of people. We want to make sure that every one of you has the same chance of entering, so we’ll be starting an admission phase soon, where you’ll be able to apply for the alpha. To make sure you can only enter once and people can’t game the system, we’ll likely not make it a random raffle, and we also don’t want to force you to follow our social media to participate, a practice that is sadly too common these days. We’ll notify you when the admission phase starts, so stay tuned!
Of course, hosting a server costs money. We want everyone of you to be able to play completely for free, with no subscription and no hidden costs, and we want the server to be entirely noncommercial. To cover our hosting costs, we’ll be opening a patreon, where you can optionally donate to support our project. We’ve been asked about a donation option before, but we didn’t want to accept donations until we could actually provide you with a running server. With the upcoming alpha we can fulfill that promise. I want to explicitly state this though: the server will always be free-to-play, and donations will always be optional. Donators will not have better items that would give them an unfair advantage in the server. However donators will get a cosmetic in-game shirt as a thank-you, similar to beta testers back in the original game.
Update: Some of you have voiced concerns regarding donation items opening up a slippery slope towards something similar to freemium and microtransaction models. We never intended for this to be that way and are strictly against such systems. However we understand your concern. To make our stance on this absolutely clear and leave no questions open, we have decided not to give out any ingame items at all for donators. Rather, items will only be available through gameplay. We still want to thank everyone who considers donating to us to help us with the server, but we want to make clear that the playing experience is the most important thing to us.
It’s been almost four years since this all began, and we’re finally getting to the point I’ve been looking forward to all this time. I’m very excited for these next steps. They’re the last parts in this quest to make this vision reality.
See you all in-game!