Progress report: October 2017 – January 2018

So, by now it’s been three months since the alpha started. Time for some updates on what’s happened in the last three months.

You’ve probably noticed that there haven’t been too many updates on progress lately. That’s partially because I’ve been pretty busy and haven’t had as much time in the last months, but also simply because alpha progress isn’t as easy to demonstrate as feature implementation progress. There’s been a bunch of improvements since the alpha started, but most of them have been minor or internal, and not the typical stuff we can show in our videos. However, by now there are enough that I can talk about them in more detail.

So here we go, a comprehensive list of bugs that were resolved and features that were implemented since the start of the alpha:

(Issue titles usually appear as the tester submitted them)

October

  • Pyraknet: Issue: Duplicate detection not working properly.

    Pyraknet is the network library implementing the RakNet 3.25 protocol that provides the low-level network layer that LU uses for network communication.
    It needs to resend packets when the client doesn’t receive them for some reason, and it needs to be able to detect and throw away resends it doesn’t need when it did receive the packet.
    In a rare case (only 1 message used by LU was using this packet type) the duplicate detection didn’t work. This patch fixed this problem.

  • Issue: Block Yard doesn’t work.

    Most objects on Block Yard didn’t appear for some reason. It turned out this was due to a change in the server’s spawning logic, which, ironically, actually was implementing a new feature, but invalidated an assumption made by Block Yard’s world script. The issue was quickly fixed with some changes to the script.

  • Implemented: Achievements for collecting powerups.

    A rare achievement type, used only by 3 achievements in the whole game, was implemented after a tester noticed the achievements not working properly.

  • Issue: Rocket Take-off uses another rocket than the one placed.

    Selecting a rocket using the mouse rather than pressing shift wasn’t working correctly, and got fixed.

  • Issue: Imaginite isn’t taken by mission.

    The mission in Nimbus station where you can trade imaginite for backpack space wasn’t removing the imaginite properly. Fixed.

    November

  • Issue: Turret quickbuilds don’t despawn.

    Avant Gardens’ robots drop turret quickbuilds, which only despawned when built by a player. Most of the time the players didn’t build the turrets and only smashed the robots, leading to quickbuilds piling up on the AG battlefields. Fixed by introducing a self-destruct timer.

  • Issue: Imaginite isn’t taken by interactables.

    Similarly, survival and wishing wells also didn’t remove imaginite on interaction. This also got fixed.

  • Issue: Survival enemies don’t spawn.

    Similar to the Block Yard issue, spawning logic was also affecting AG Survival. Similarly fixed with alterations to the script.

  • Issue: Unimplemented achievement rewards.

    A few achievements didn’t give out rewards when completed. It turned out this was due to some entries in the game’s original database not being set correctly.

  • Implemented: Achievements for survival times.

    Most achievements regarding survival times were already implemented, but it turned out the game actually uses two separate mechanics for these achievements, and only one of them was implemented before.

  • Issue: Monument finish line won’t end quickbuilds.

    A quickbuild wasn’t completing correctly. It turned out this was because the object overrode the game’s database on quickbuild complete times. Fixed by implementing the override.

  • Issue: Imagination fountain doesn’t drop imagination.

    Some items that were supposed to drop powerups, didn’t. Fixed by implementing the associated script.

  • Pyraknet: Issue: Absolute time used when relative time needed.

    In some packets pyraknet has to send timing information in packets. It turned out the time it was sending was absolute on Linux (but not on Windows), when it should have been sending time relative to program start. This change enforced using relative time everywhere.

  • Implemented: Random missions.

    The game features some daily missions that are drawn from a random pool and not offered sequentially. Support for this was implemented.

  • Issue: “Stagecraft” achievement not working.

    Another bug caused by a script not working correctly. While some more work on the script remains to be done, it now correctly updates the achievement.

  • Issue: Mission to play a guitar not working.

    Similarly caused by a script, and similarly had some complications appearing that caused full implementation to be postponed. The mission is correctly updated now, however.

  • Issue: Paradox’s Plasma Ball 1 does not kill enemies.

    Caused by a bug in skill parsing. Skill structures remain relatively unknown, mostly due to complex conditions that are hard to reverse engineer. A few similar bugs for other items still exist, and in some cases investigations are pretty much stuck because the condition can’t be analyzed even when the matching code is found. In this case however it was possible to resolve the bug.

  • Issue: Plasma Ball 1 jump attack does nothing but consume imagination.

    In some instances, skills can cancel their execution, as happened with the item here, which doesn’t cast its effect if the player is jumping. The server was however still removing the skill’s imagination cost, even if the skill cast had failed. This was fixed to only take the cost if the skill actually succeeded.

  • Issue: Spider Cutscene being played for everyone

    Some network packets are supposed to be sent to all players, while some should be sent to just one player. In this case, it seems a message that would normally be sent to all had its behavior overridden to only be sent to one player. The server didn’t know about the override, and sent it to everyone, causing the cutscene to appear for everyone in the world. Fixed via override.

  • Issue: Rocco Sirocco doesn’t spin

    A script wasn’t working properly, and wasn’t displaying a cinematic when the mission completed. Fixed through script modifications.

December

  • Issue: Daily Mission “Six Shooter” doesn’t give Plunger Gun.

    A script wasn’t implemented, and didn’t give out a mission item.

December featured several large refactors, which reorganized the server internals and cleaned up the code. They didn’t change the external behavior of the server by much, but nonetheless represent significant improvements, as they affected about half of the server and made it easier to implement new features in the future.

  • Refactor: Server command system.

    The server command system was rewritten to be completely separate from the core server. As a result of this change it’s now possible to delete the entire folder related to commands, and the server will continue to function as normal (just without commands, which aren’t part of normal gameplay).

  • Implemented: Mail attachments.

    Mailing attachments had been supported for a while, and the server was already using it to distribute achievement rewards when inventory space was full. However the interface allowing players to send items per mail was disabled, until this patch changed this.

  • Implemented: Mail cost.

    Sending mail costs coins, and mailing achievements increases this cost by the cost of the items mailed. With this change, the server now subtracts the correct cost from the player when sending mail.

  • Pyraknet & LU server: Refactor: Separate BitStream into read-only and write-only versions.

    Pyraknet also provides a way of reading and writing binary data sequentially, and which allows reading/writing single bits. It turned this was almost never used for reading and writing from/to the same stream, and therefore could be separated to make the intent of the stream more clear.

  • Pyraknet & LU server: Refactor: Change to use composition instead of inheritance.

    Pyraknet had previously been using an inheritance model, where the LU server overrode certain things to implement its own functionality. This worked well and allowed for a lot of freedom for overrides, but didn’t separate the low-level pyraknet stuff from the high-level LU server stuff well. This was changed to use composition instead, where the LU server interacts with the low-level layer mainly via callbacks and delegations, which provided proper separation.

  • Pyraknet & LU server: Refactor: Add type annotations.

    Python is dynamically typed, and as such doesn’t enforce a variable to remain the same data type. However information about the intended data type is useful for documentation and static validation, and so Python has added support for this information in recent versions. With type annotation support improving, it was time to add type information to the pyraknet & LU server projects where it hadn’t already been present.

January

  • Pyraknet & LU server: Refactor: Various refactors.

    A bunch of minor things were changed and improved, some variables were made more restricted, and some internal systems were reorganized. The changes are relatively specific to the internal implementation, so I’ll keep this paragraph short. However, that doesn’t mean these changes were insignificant – grouped together, they were one of the larger refactors.

  • Issue: Buttons at the AG Monument trigger even when not built.

    Buttons could be pushed even though they hadn’t been built yet, also known as “invisible buttons”. This was caused by an unimplemented check in the component, and was fixed by implementing the check.

  • Pyraknet & LU server: Refactor: Restrict bitstream interface further.

    Previously it was possible to access packet data both by array index notation and via the sequential interface. In practice the random access was rarely used, so this patch removed this functionality and restricted the interface in some other points as well, which makes passing the bitstream to other handlers safer.

Next up

I hope this list could give you some insight to the kind of bugs that pop up in alpha testing, and to what I’ve been working on in the past months.

There have been a bunch of bugs fixed since the start of the alpha, but there’s plenty left to fix. 50 bugs remain open, and while most of them should be possible to fix, it will take some time. I want to make sure the server doesn’t just have a lot of worlds, but also runs smoothly, and for that to work, it’s necessary to fix the open bugs before opening the next alpha stage. Based on the time it took to fix the above bugs, it’s likely it will take some months until the server is ready for the next alpha. But by then, the server should be pretty robust. 🙂 I’ll publish something when the server’s ready, and I’ll continue the progress updates in the meantime.

See you all in-game!
-lcdr

 

Alpha selections

Hello everyone!

It’s been a week since the application phase ended, and now it’s finally time to announce the results.

First of all, thanks to everyone who submitted an application. In total, we received 303 applications, more than I’d ever have expected. I’m really impressed with how many people are still around who want to play LU, even though the game closed 5 years ago, and how many would apply even though this is just a first alpha. I went ahead and read every one of those 303 applications you sent us, which took a couple of hours, but I felt it was the least I could do to honor your dedication to this game. Your applications were much more than I imagined, with many of you detailing fun moments in original LU, your enthusiasm for the game, and your excitement for being able to play again. It was a great and inspiring read, thanks to you all for taking the time to write all this.

The application system also seems to have worked well to prevent spam; out of 303 applications only 1 was spam (and don’t worry, we didn’t accidentally classify your application as spam, the application in question was a clear impersonation we manually confirmed before disqualifying it).

Now, before continuing with the selections, a note on how the system works: You submitted your application in the application phase. From these 302 applications we selected 20 to be included in the first alpha stage starting on October 26th. The selections were made randomly, using a computer (to be precise, a small python script). However, these 20 people are just the first stage. To evaluate the server performance under load, we’re starting with a small amount, and will add more people in later stages. This also means that if you submitted an application and didn’t get selected, you still have a chance, because your application also counts for the later stages. If you haven’t had a chance to submit your application, you will also still have a chance in later stages, where we will be accepting new applications as well.

Now, on to the selections!

We’re selecting 20 players out of 302 applications, which means if you entered you have a 6.62% chance of being selected.

The selected players for Stage 1, starting on October 26th, are:

 

  • Komplexus
  • Dazoo1
  • ethanlindley
  • Tinka
  • Baffish
  • lordpickleking
  • Homie0Jr
  • emanresU
  • Liam3997
  • Astrophel
  • mater06
  • Mickeystick
  • Halofan4566
  • Matthew
  • LS2112
  • Raydiation
  • DocGrognard
  • creeperkid75
  • stefanoss
  • ShiWham

Congratulations to all of the above. I’m glad the random algorithm provides for so much diversity. We’ve got all kinds of LU fans in there, from original beta testers to live players, to FTP players, to people who never got the chance to play, people from the developer side and people who are engaged in the community, young ones and AFOLs who played LU with their kids. I’m really looking forward to all of them coming together in-game to play LU again.

If you’re one of the selected, check your mail, we will provide you with info on how to connect later today.
If you’re not selected you may still have a chance in later selections, as mentioned above.

And no matter whether you’re selected or not, we will continue to share info about our progress and the alpha with you. You’ll be able to look forward to alpha gameplay videos not only from our youtube, but from the players as well, as we allow streaming and recording of the alpha.

Remember, this all starts on October 26th! I’ve been looking forward to this next step towards everyone being able to play again for years now, and I’m excited that it’s only 5 more days until then.

See you all in-game!
– lcdr

 

 

Alpha release date & timeline

Hello everyone!

We’ve done a lot of server testing over the last month. I’m glad to say that we’ve been able to fix the bugs that came up, and implement the necessary server administration features. We believe we’re ready for a first alpha, where we will invite people from the community to test our server.

The first alpha stage will begin on October 26th, 2017.

We’ve mentioned before that we’d like the alpha selection process to work in a fair way, and we’ve come up with a process that we think will help with this.

We’ve decided not to base alpha access on giving out keys, but instead via applications. Everyone who wants to join the alpha can submit an application describing why they want to join, and after filtering out spam we randomly select from the submissions. This will help with duplicates, and complications with people asking for keys.

We’ve decided to start low with 20 slots available for the first alpha stage. This is so we can analyze how well the server responds to multiple players playing simultaneously, and possibly optimize the server to work more efficiently under these conditions. After we complete our analysis, and implement bug fixes and optimizations, we will expand the alpha in later stages to include more players and features.

To explain the process, here’s a timeline:

September 30 (Today) Alpha application phase starts. From now on, people are able to submit their application for alpha.
October 14 Alpha application phase ends. People will still be able to submit applications, but they will count towards a later stage, not the one beginning on October 26.
October 21 After filtering out spam and randomly selecting applications, we announce the selections on October 21. Selected applicants will be notified via mail about the next steps.
October 26 Alpha Stage 1 starts, with 20 selected people.
Later Once testing with the initial group of 20 people is successful, we will open up the alpha to more people in later stages.

We’ve written up some more detailed explanations in our Alpha FAQ. Check it out if you’ve got a question, or post in the forums.

You’ll be able to submit your applications to the first stage starting now, until October 14. Click below to start your application for a chance to be selected.

Apply for Alpha

We’ve been wanting to do an alpha for a while now, and we’re glad we’re now able to start the process. Check back on October 21 when we announce the selections.

See you all in game!
– lcdr

Progress Report: August 2017

This is just going to be a pretty technical short summary about what we’ve been doing this month. Don’t expect any big announcements here, think of it more as a “behind the scenes” look at what we’ve been working on. Server development is just one of multiple things we do, so there’s more than that listed here.

Note: Before I start, I’d like to mention something:
In the last post I mentioned that for server costs, we’re going to have to set up some sort of donation system at some point. I also mentioned that we’d like to thank donators in some way, and suggested an in-game cosmetic shirt for this. Some of you have responded that you see this as too similar to “freemium” or “pay-to-win”. We only planned this as a small thank-you, with nothing more in mind, similar to something like donator mentions on twitch. However, we understand your concern, and I want to assure you that we will never be “freemium”, everything is and will be 100% free, no catches. Therefore, to leave no questions open, we have decided not to give out in-game items at all. Rather, items will only be available through gameplay. We still want to thank everyone who considers donating to us to help us with the server, but we want to make clear that the playing experience is the most important thing to us.

Now, on to the actual progress report!

Research

  • Investigations on older lvl formats. Lvl files are used by the game to store world information like NPCs and Enemies, and are vital for a server to work.
  • Investigations on the game’s audio formats.

Research tooling

  • As a side effect of the above investigations, it turned out the current structure definition language used to document formats is not completely capable of describing all formats. Currently missing are
    • Support for pointers in formats
    • Support for recursion
    • Support for definition of larger structures from smaller ones
    • Complex looping

This needs to be added at some point to be able to describe more complex formats. The struct definition language is used by the captureviewer and structparser tools to parse binary files and packets. Luckily so far it has sufficed for them, which is why these issues have only surfaced now.

  • Work on a struct-visualizing hex-editor to help with internal research on formats. Screenshot:
    Parsed and highlighted stucts in the viewer, with unparsed part visible below.

    It’s still experimental and too early to release, but it’s helpful for investigations on file formats.

  • Work on the client’s fdb database format.

Documentation

  • Work on cataloging available clients. For research purposes we are always interested in clients from older versions, especially from beta and alpha.
  • The docs are now also available in a faster-loading read-only mode.
  • I’d like to do some more documentation maintenance at some point, but it’s difficult to find time.
  • Work on improving contribution-friendliness to the docs, in cooperation with researchers from other projects. We always welcome contributions, especially if they are well-researched and -sourced.

Development:

Pyraknet

Pyraknet is a minimal port of the network library used by LU, RakNet 3.25, to python. It’s open source and used by multiple projects.

  • Work on making the library more robust, featuring refactors, tests and type annotations.
  • Implemented support for split packet receiving. This was not possible before due to having no proper way of getting the client to send large amounts of data.
  • Investigations on maximum packet size. It seems LU has a packet size of 1200 bytes hardcoded for some reason, even though it’s not really necessary in the network. But just to be sure pyraknet will also use this lower size from now on.

Server

  • Features seen in our video for this month. Side note: LU is bananas.
  • Ongoing work to test the server against edgecases, high loads, network issues and others. This will take a lot of our time, so it’s pretty likely we won’t be able to release a video next month.

Website

  • The website has been online for a month now, and everything’s working well. Thank you for your threads in the forums, I’m glad the atmosphere is so nice there.
  • And of course thanks to everyone commenting on twitter and youtube as well. I read all of your comments, and I really appreciate them 😊

Notes on Alpha

It will still take a few months to prepare everything for an alpha release. Please be patient until then. We’ll post updates about release and admission phase dates once we have something to report, so you don’t have to worry about missing them.

 

See you all in-game!
– lcdr

Site Launch!

We’ve finally gotten around to launching a website. So far we’ve focused on server development, but now we’re ready for a public-facing display. There’s project info, FAQ, media and ingame screenshots, and what I’m looking towards most, forums, which brings our project one step closer to being a complete community. And of course there’s also this dev blog, where we’ll be posting longer updates & announcements, as well as some write-ups about the internals of LU. (For shorter updates check our twitter as well).

Got suggestions? Something not working? Leave a comment or post a topic in the forum.

Plans for the future

The launch of this website clears another checkmark off our list towards a first alpha release. We believe the server is mature enough at this point, and have made sure that the features are working well. We’re working hard to deliver to you the server you deserve, and the first step in that direction is an alpha test where we will host a small server instance to a limited set of people. We want to make sure that every one of you has the same chance of entering, so we’ll be starting an admission phase soon, where you’ll be able to apply for the alpha. To make sure you can only enter once and people can’t game the system, we’ll likely not make it a random raffle, and we also don’t want to force you to follow our social media to participate, a practice that is sadly too common these days. We’ll notify you when the admission phase starts, so stay tuned!

Server costs

Of course, hosting a server costs money. We want everyone of you to be able to play completely for free, with no subscription and no hidden costs, and we want the server to be entirely noncommercial. To cover our hosting costs, we’ll be opening a patreon, where you can optionally donate to support our project. We’ve been asked about a donation option before, but we didn’t want to accept donations until we could actually provide you with a running server. With the upcoming alpha we can fulfill that promise. I want to explicitly state this though: the server will always be free-to-play, and donations will always be optional. Donators will not have better items that would give them an unfair advantage in the server. However donators will get a cosmetic in-game shirt as a thank-you, similar to beta testers back in the original game.

Update: Some of you have voiced concerns regarding donation items opening up a slippery slope towards something similar to freemium and microtransaction models. We never intended for this to be that way and are strictly against such systems. However we understand your concern. To make our stance on this absolutely clear and leave no questions open, we have decided not to give out any ingame items at all for donators. Rather, items will only be available through gameplay. We still want to thank everyone who considers donating to us to help us with the server, but we want to make clear that the playing experience is the most important thing to us.

It’s been almost four years since this all began, and we’re finally getting to the point I’ve been looking forward to all this time. I’m very excited for these next steps. They’re the last parts in this quest to make this vision reality.
See you all in-game!
– lcdr