You should take a look at the hints provided by the audit tool.
The audit breakdown has much higher value than the index itself. You will see what affects the performance of your page and can evaluate if you can do anything about it.
It's clear that an empty page will have excellent performance. As soon as you start adding content and features the index will get worse. Adding features will always require additional resources. For example, the DayPilot library has a few hundred kilobytes (the current version is almost 800 KB uncompressed). You can't do anything about that - it's the price for getting access to the features.
However, the performance audit will suggest improvements that will help you overcome this. With regard to loading the library itself, it will include things like:
1. Move the <script> section from the header to the body, preferably to the bottom (but not above code that uses it). That will let the browser load the page without waiting for the script and it may be able to display something to the user while the script is loading.
2. Make sure that gzip compression is enabled for the js file on the server. This will reduce the size to about 200 KB.
3. Make sure that the HTTP headers allow client-side caching of the js file. Use a long expiration time (a week or more). This will let the browser cache the file without reloading it every time. You can enforce reloading of the script by appending a version-specific query string.
You might see more hints in the audit results.
For your reference, the main scheduler demo page (https://javascript.daypilot.org/demo/scheduler/) has a performance index of 74. And there are no special optimization techniques applied (the script is loaded in a blocking way in the <head> section, for example).