Aug 142017
 

Finally, we turn to Philipp, who is working to upgrade the Kodi windowing system in Linux from X11 to Wayland. As a PR is already posted, we are extremely excited about this project!

Wayland Support – yol

My main goal was quite simply to get Kodi to natively support Wayland again, if possible as well as it currently supports X11. The Wayland protocol is slowly but steadily gaining traction in the Linux world as successor to the X11 display server and is generally seen to replace it in the long term, which is why it is essential for Kodi to support it if it wants to continue to deliver a good user experience on Linux. As some may know, Kodi did already have a Wayland windowing implementation originally written by Sam Spilsbury during GSoC 2013, but it got removed from master for unsolved infrastructure reasons some time ago and was never on-par with the X11 implementation feature-wise.

June

I subdivided the main goal into a number of sub-goals/features in my initial proposal categorized as “basic” (i.e. necessary so you can reasonably use it at all), “additional”, and “possible extension”. In the first month, I dealt with all basic and some of the additional features. I had planned to salvage as much of the previous code as possible, but different design choices I made ultimately meant that I ended up writing most of it from scratch. Still, I was frequently checking the prior implementation to see which parts of Kodi I have to touch and how I can go about solving some specific aspects.

To get a bit more concrete here, where the approach differs most is how the Wayland protocol objects are used in object-oriented C++. Previously a home-grown approach was used, probably because there were no C++ bindings for libwayland-client at the time. The resulting wrapper code is quite tedious to write and maintain, which is why I decided to use the C++ waylandpp library as base. Even though it is not quite mature yet, I figured that my time would be better spent improving waylandpp than trying to come up with something new on my own. It is of course possible to use the C API of libwayland-client directly without any wrappers, but this is not a very nice solution since the Wayland protocol is really made out to be used in an object- oriented fashion if the language allows for it. Much to my delight, the author of waylandpp was very responsive and supporting, which means that all improvements and feature additions I made to waylandpp at this point are already merged upstream.

Unfortunately, it soon became clear that the aging windowing infrastructure of Kodi is not a good match for Wayland which does some things in a very different way. Wayland is designed such that clients have minimal control over the desktop for security reasons. This for example means that, unlike on X11, Windows, and most other windowing systems, applications cannot just set arbitrary video modes. They can only tell what their preference would be to the compositor, which then decides and in turn tells the applications what size they get. I do think that this is a good design decision since it solves a lot of problems that XRandR has like leaving the desktop in a bad resolution when a program crashes, but it does mean that a lot of things have to be done differently. The Kodi code assumes the traditional procedure of fetching a list of valid resolutions and then being able to switch to any resolution immediately and basically without failure. Changing this in a satisfying way would require quite a lot of refactoring that would have to touch all of the current windowing implementations, too. This is a lot of work, requires coordination with all platform maintainers, and is not realistic to be completed within the GSoC timeframe, so I have put this off for now and instead tried to make minimal adjustments to the Kodi resolution switching code that allow the Wayland implementation to work without impacting any other windowing system. The downside to this is that the Wayland code is not as clean as it could or should be and that there is an increased risk of weird stuff happening.

July

After getting most of the basic stuff out of the way in June and the first week of July, I focused on the two big remaining issues: Support for the wp_presentation Wayland protocol extension and windowed mode, the first of which was just merged this week. Put simply, Kodi now has a reliable way of determining how much time it takes from rendering a video frame until it appears on screen and does not have to resort to guesswork, improving AV sync. Also, it can be used as source for the video reference clock so that slight mismatches of the video fps with the screen refresh rate can be corrected by resampling the audio.

I worked on windowed mode support for the remainder of the month. I guess that it would not usually be expected to be a complicated issue, but most Wayland compositors require applications to draw and handle window decorations themselves (called client-side decorations) if they want to have any. I discussed with my mentor how to go about this and we decided to once again keep things local to the Wayland implementation and draw the decorations there as none of the other windowing systems currently requires this. The other option would have been to somehow integrate the decorations into the Kodi skinning system. This is of course nicer, but far more complicated and probably not worth the trouble. As media center, most users will use Kodi in full screen mode which is also the default, so window decorations that don’t look that nice will not be a big problem.

During the past two months I also had to learn that Wayland is still quite new to the scene and that a lot of bugs still lurk in the various components. I had to report and sometimes fix bugs in compositors and other third-party components such as mutter, weston, and libva. Finding these bugs and determining whose fault it is that something did not work as expected took more time than I had initially anticipated. I would like to thank the #wayland IRC community and specifically pq and SardemFF7 for their time and advice that helped me move forward several times.

August

In the last few weeks I finished windowed mode, investigated some outstanding issues not directly related to Wayland such as touch screen handling, and am now trying to get my work merged mainline. I do hope that no big issues pop up that could prevent the merge, but I’m optimistic – the pull request is already live [https://github.com/xbmc/xbmc/pull/12664]. Apart from fixing the outstanding review comments, I plan to look at identifying possible memory leaks and multi-threading races with valgrind and ThreadSanitizer.


GSOC 2017 Update – Wayland Support
Source: Kodi

Aug 112017
 

It has been a while since we released 17.3 release where we fixed several issues. Now the time has come to do another where we tackled several more issues that were identified. Although we already moved on with development towards v18 we do take the time to fix issues for the current release when we can. To give these fixes a proper test run before we call it final we first want to make this release candidate available for the wider audience who might be facing some of these issues we have fixed. Just browse the list below and give it a try.

Fixes done in this release:

  • Potentially fix crashing on Windows due to an issue in Python
  • Potentially fix crashing on Windows when enabling zeroconf
  • Fix sporadic crash on Windows when installing or updating add-ons 
  • Fix issue for users with reverse proxies attempting to forward websockets.
  • Fix possible issue if Linux distro uses system ffmpeg and cause black screen with 10-bit H.265
  • Properly throttle scraping music information online to prevent overloading the provider
  • Fix native keyboard on iOS 11
  • Fix potential crash on Android O loading App icons
  • Fix non showing Kodi banner on Android O
  • Fix potential crash on Android with certain keymaps
  • Fix wrong detection of VP6 and VP8 videocodec on Android
  • Update FFmpeg to 3.1.9
  • Set hard requirement to use FFmpeg 3.1.x only
  • Fix for Hangup when viewing recording and pressing next/previous
  • Fix merge scraped album type and label correctly with that derived from tags from music files

What else is new?

In the bugfix releases we never include any new features. They are as feature complete as the initial version with the difference is they contain stability and usability fixes. If you are curious you can read up on all the v17 changes here: Kodi v17.0 “Krypton”

Where can I download Kodi?

As alway you can find the official builds on our download page. Then click on the platform of choice and choose pre release build. You can install these build just on top of your current Kodi installation without doing a reinstall or cleanup as we do a full migration if needed. All you add-ons or installed skin will keep working.

Apparel, donations or getting involved

Getting involved is quite easy. We encourage you to report problems with these builds on our forum first and after that, if asked, submit bugs on Trac (following this guide: How to submit a bug report). Do note that we need detailed information so we can investigate the issue. We also appreciate providing support in our Forums where you can. You can of course also follow or help promote Kodi on all available social networks. Read more on the get involved page. We are always happy to receive a donation by which you show your support and appreciation, and t-shirts and Raspberry Pi cases may still be found on the sidebar for purchase. All donations and other income goes towards the XBMC foundation and are typically used for travel to attend conferences, any necessary paperwork and legal fees, purchasing necessary hardware and licenses for developers and hopefully the yearly XBMC Foundation Developers Conference.


Kodi v17.4 RC1: Just a bunch of fixes
Source: Kodi

Aug 012017
 

Next we have Arpit’s work upgrading Kodi’s add-on backend from Python2 to Python3

Python3 Support – arpitn30

My initial proposal was to support both Python2 and Python3 in Kodi by maintaining two versions of the same libraries and calling the required one depending upon the meta information in the XML files of the add-ons.

JUNE

During the first week of June, I was mostly understanding how everything was being built and what needs to be changed. I initially thought of updating the input to swig so that it can output python3, but I realised that Swig only changed the code to generic ML code which was given as an input to Groovy which with the help of Python Templates outputs Python code. So I cloned the Python templates directory and in the later weeks, I started to study and gradually change the python templates to that of Python3.

Aside from the major unicode string change, Python 3 changed a lot of things at the base level that drastically affected the Python-C API which was used to build the templates. So I went through each and every function and decided if it should return a unicode object or a byte object. I changed all the Python API function names, the module initialization, the Py_TPFLAGS, and changed the char data type in C to wchar data type. I also looked at all the data type which were being converted from C to Python and Python to C and inspected if any changes to data types in Python 3 like ‘map’ returning and iterable instead of lists and also changed the PyInt type which was no longer being used in Python3 API.

During the mid of June, when I started working to support both versions of Python, I was told to test if my code supported was even working with Python3 or not. So I forked my current working branch into 2 branches, one to test if my Python3 templates work correctly (testpy3) and the other to support both Python 2 and 3 (build2n3) and was working on them simultaneously. I also updated the HttpNetworkHandlers to Python3.

In Build2n3 branch, I changed paths of the Python libraries to the new Python3 directory, updated the Cmake files to download dependencies for both and keep them seperate.

In testpy3 branch, my main focus was to run the environment with as minimal changes as possible. So I removed 2 python template directories and made python3 the main directory. I had to change all the cmake files though to the updated dependency structure. After all the kinks had been removed Kodi was able to run with Python3 at the end of Phase 1 but the add-ons weren’t working yet.

JULY

The first thing I needed to do after the end of Phase 1 was to get the Python3 version of Kodi working, so that I could know if my code was working or not. The major trouble was that there wasn’t any way to know the error except the for only Log Entry of PythonModules not found. I skimmed through each and every documentation to figure out the changes which were made and the difference between them.

I found that the threads and GIL in Python3 are implemented differently than Python2. The acquirelock and releaselock functions are now deprecated and had to be changed with acquirethread which needed to have a saved state beforehand.

But the problem about the threads was still not solved. After a lot of testing and help from Paxxi and Rechi, we were able to identify the problem which was due to the duplication of state of _PyThreadState_current by Kodi and Python Bindings which resulted in the thread state being lost mid execution and we solved the problem by changing the cmake file in templates and adding the python binding directly to the xbmc core library.

After that, I started working on Build2n3 branch to apply the changes which were done in testpy3 branch and to figure out if it was possible to create two versions of libraries with the different implementation of threads even though it meant building Kodi twice while I was still looking into the ModuleNotFound error in the other branch which was still not resolved.

So I started working on Build2n3 branch. The problem I was having was with the env variables like PythonLibs and Python_include_dirs which can be only set to one of the versions at a time so we had to unset the variables after they were used and set them to the python3 directory. 

The main point now was if it was practical to go for a version that supports both Python 2 and 3 or skip a version and release just a python3 version whle giving the developers time to port. Changing a major amount of code just to support Python 2 and 3 only to revert it back to the current code one or two versions later seemed impractical if not impossible due to changing the implementation of handling of threads. Taking in account the time it took, just to find the bug and get just the Python3 version right, Razze asked me to focus on just Python3 version for now.

So going back to testpy3 branch, after searching for several days, we found out that the function which initialized the modules did not add it to the built-in modules in Python3 which wasn’t mentioned in most of the Python3 Porting guides. So we added statements to explicitly add it to built-in modules. And finally the ModuleNotFound error was gone.

After fixing a few bugs like correcting the charset conversion to wchar, updating the execution script, Python 3 Kodi was finally working with the built-on Kodi libraries being perfectly imported. 

In the next week, I checked the best way to make Kodi add-ons compatible with both versions of Python either by using __future__ packages or by using sys.major_version condition and I decided to go with the latter. I also made the versioncheck addon compatible with Python3 and added a pr to it. The addon was working perfectly with current version of Kodi as well as the Python3 version of Kodi.

Lastly, I decided to update the Python3 branch with the changes which were later made in the templates while working to support just Python3. So if we decide to go for a version to support both Python2 and 3, we can just fork from there. 

I also added the PyFile function into Kodi itself thus removing the dependency on Py3c fileshim.h.

I also tested the debug and release versions of Python3 Kodi on Windows 64 bit and it worked without any problems.

The Future

I’ve tested my code with Ubuntu using the natively installed packages and it seems to work. And after a few days, I’ve been able to build the master branch in Linux using the target packages only. I intend to finish updating the files for target version for Posix systems by the end of next week. I will port some of the add-ons to Python3 later on, and I need some time to clean up the code as well. Although I removed the commented code which I left behind from Python2, I still have to give it a final review before submitting my code. 


GSOC 2017 Update – Python3 Update
Source: Kodi