Question: Epic: Implement remaining JetBrains IDE's (CLion, WebStorm, Rider, RubyMine)?

It should be in `.gitpod.yml` if we create a dependency from the init task.
Would it as an intermediate solution be ok to simply download the binaries (in docker or init task) instead of configuring the IDE on project level?

Epic: Implement remaining JetBrains IDE's (CLion, WebStorm, Rider, RubyMine)
We are currently only supporting the following JetBrains IDE's
- [x] GoLand
- [x] IntelliJ
- [x] PHPStorm
- [x] PyCharm

However, we need to add remote development backends support for:
- [x] CLion ( URL
- [x] Rider ( URL
- [x] RubyMine ( URL
- [x] WebStorm ( URL

Related:
- URL

 Front conversations

**Callout:** 👋 Are you interested in this feature? We would be interested to talk to you to hear your feedback.

Internal conversation [1]

I think it depends on URL Till we cannot do it automatically with smoke testing it will be tedious to review so many PRs.

Is there any expected time of arrival for other IDEs support?

Gitpod current focus is on the IntelliJ experience (which has a lot of overlap with the other JetBrains IDEs), which is also where JetBrains themselves are focussing a lot of their remote development energy. As the experience improves for the other IDEs we'll be looking to roll those as soon as we reasonably can 🙏

With URL done, we can simplify our build process to add more components easier.

Hey folks, wanted to give a quick update (as many have asked!). Firstly, thanks so much to all our users who have given feedback in the community for our JetBrains integration. Please continue to do so, we're learning a lot from you and we're able to refine the experience based on your input.

If you're late to the party, I wanted to give a little background and context into Gitpod IDE support and why we've not yet released full support of the IDEs just yet. For background, we have a partnership with JetBrains. In real terms is that we have conversations with the friendly JetBrains people to help input on the future of JetBrains Gateway as a whole and help create a really great remote dev experience with JetBrains in Gitpod and beyond!

Opening up an IDE is only one step in the process to bringing a great developer experience in Gitpod. Whilst a lot of the Gitpod functionality is shared across the JetBrains IDEs, such as prebuild support, integration of Gitpod task terminals, port forwarding and port behaviours. There are also lots of language specific product changes that need to be made in both Gitpod and JetBrains.

**For example:**
* Ensuring coverage of plugins / VS Code extensions for the language - see recently added Oracle support in OpenVSX
* Our documentation support is good - see recently updated Java page
* Integration with build tools for the language - we are working on Maven right now, etc).

For that reason we've been a bit deliberate in how (and when) we roll out other IDE support. We don't want to open up an IDE for our Gitpod users to go try the integration, but then be dissapointed (let's say if some aspect of the integration is not up to the standard they expect, some plugin doesn't work, etc).

**In simpler terms: The quality of the experience is really important to us.**

We just opened up WebStorm and RubyMine (yet to be announced). So hopefully we will get to the remaining IDEs in the coming weeks ! Come join us on Discord, chat about your experience and give us your feedback (we are listening!). We're super excited for what's to come with JetBrains + Gitpod 🙏

- Rider + CLion docs published:
- Changelog in draft, will be published soon

Closing as complete ✅

Epic: JetBrains Prebuilds for Maven
We haven't yet investigated this, this ticket is a placeholder for any changes required on our side to make indexing easier for IntelliJ and/or document any processes that improve the performance.

**Remaining**

- [x] Move warmup login in JetBrains Launcher
 - [x] Update JetBrains Launcher to support warmup command [[1]]( URL
 - [x] Move warmup logic from ide-service to JetBrains Launcher [[2]]( URL
 - [x] deploy and rebuild all stable and latest images (re-run nightly jobs)
- [x] switch 100% of traffic to new ide-service resolver workspace config endpoint
- [x] decouple JetBrains Launcher from JB IDE images
- [ ] Remove warmup logic from ide-service.ts in server

**Completed**
- [x] Allow layering multiple JetBrains products in one image

**Value & measuring the impact**
- [ ] Sync with Dev Rel on some possible announcements and content or blog posts (e.g. blog post showing working with a single project like petclinic for Maven. Show some performance before/after benchmarks and talk about the relation and relevance to CDEs, Gitpod Prebuilds, etc)

### Considerations

* **Performance and cost implications** - Ideally we need to find a way for a project to opt-in to indexing, without burdening the user with too much (or any configuration) if we can.
* **Backends included in prebuilds** - Currently backends are loaded on top of an existing workspace, not during pre-build, so for the indexing to be done before the workspace starts we need to have the backend executed within a prebuild.

**Related issues:**

- URL
- See internal link from discussion with Jetbrains
- Also see internal thread from Gitpodders discussing implementation
- URL

The `remote-dev-server.sh` seems to support a `warm-up` command (similar to `installPlugins`) which could be used.

@U1F984 yes, but there are several issues:
- desktop IDEs right now are not running in prebuilds
- index is not stored in /workspace folder, so it is not cached and does not survive restart

If it is unfeasible to store the system directory in /workspace maybe Shared project indexes could be used. However this is an ultimate edition feature.

@akosyakov can you link to full workspace back-up issues, etc that might be blocking this issue?

System config and cache directories were moved under `/workspace` folder as a part of URL

We need to add ability to include JB backend into prebuilds. There following things to consider:
- indexes created by one backend is mostly not reusable by created by another backend, so there should be a way to configure for which backend indexes should be created per repo/project
- there are different places where we could configure it, like in .gitpod.yml or in settings of project.

As a prototype, for the start we could support only IntelliJ indexing and it in .gitpod.yml as an experimental feature.

We are currently trying out Gitpod as maintainers of URL and the IntelliJ integration is a big improvement for us a Java developer.

However, one of my main use cases is to review and try out PRs and because of the nature of our repository, IntelliJ will take 10 minutes or more to index the project. This makes using workspaces as throwaway instances for trying out things very hard for me.

So I just tried to explain our use case and highlight how such a feature would considerably unblock our Gitpod adoption 🙂

We don't support it natively yet, but I got it working in this branch: URL

Important bits:
- make sure that your gradle/maven projects have configured java target compatible with Java in your image (11 in default image) otherwise JDK indexing is stuck: URL
- add prebuild task: URL

This task downloads IntelilJ backend which MUST be aligned with URL
Then it configures to move indexes to the same directory which will be used by real backend in regular workspace and runs warm up.

If something does not work (hangs) you can run the same task in regular workspace to trace with jstack. Important thought you should use another folder, since only backend can lock the folder at the same time. Please capture then warm task logs and jstack output and share here.

We shipped an **experimental** support of indexing for IntelliJ IDEA using Prebuilds.

Adding note that we have URL - To move prebuild configurations to the project page. It might therefore make sense to move the JetBrains prebuilds configurations also to that page in the UI rather than in the configuration.

Renamed the title of this issue, it was created a long time ago when our understanding of JetBrains Gateway was just forming. We can see now the opportunity to focus on Maven for the time being to get the Prebuilds and opening IDE experience working really well. I'll scope this issue down now to just Maven. However, we'll come back and open a companion issue for Gradle also in time. Learnings from integrating Maven fully with Gitpod will then also carry over.

@andreafalzetti has been doing some analysis to compare:

* starting without prebuild to starting with prebuild
* restarting a stopped container

Will share on the issue.

Work currently paused, waiting on some input from JetBrains.


Question: macos local run fails?

**Hi there!**

Thanks for reporting this issue. It has already been submitted previously, so I’ll go ahead and link both reports and close this one as a duplicate.

I recommend following the main thread here: URL This will help us keep all updates in one place and ensure clearer communication as we work on a resolution.

Thanks for your understanding!

macos local run fails
### Can we access your project?

- [X] I give permission for members of the FlutterFlow team to access and test my project for the sole purpose of investigating this issue.

### Current Behavior

flutterflow local run gives below error : CocoaPods's specs repository is too out-of-date to satisfy dependencies. To update the CocoaPods specs, run: pod repo update

if i do pod install --repo-update then i get error : \[!\] Unable to find a specification for `firebase_core` depended upon by `cloud_functions`

You have either:

* mistyped the name or version.
* not added the source repo that hosts the Podspec to your Podfile.

### Expected Behavior

I did followed everything from internet and tried all possible upgrading , deleting , cleaning , ruby upgrade , cocoappod upgrading , even manual symlinks creation from firebase and nothing helps. I have latest M4 chip macbook , latest os : sequoia 15.5 , with latest xcode 16.4 , latest cocoapod 1.16.2 , latest ruby 3.4.4 , ios 18.5 , simulator with device iphone 16 plus. there is nothing to upgrade. And basic simple local run doesnt works

### Steps to Reproduce

1.just simple local launch doesnt work in any macos even the latest one available in the world.

### Reproducible from Blank

- [ ] The steps to reproduce above start from a blank project.

### Bug Report Code (Required)

\[!\] Unable to find a specification for `firebase_core` depended upon by `cloud_functions` You have either: \* mistyped the name or version. \* not added the source repo that hosts the Podspec to your Podfile.

### Visual documentation

### Environment

markdown
- FlutterFlow version: latest desktop APP
- Platform:MACOS
- Browser name and version:NA
- Operating system and version affected: latest M4 chip macbook , latest os : sequoia 15.5 , with latest xcode 16.4 , latest cocoapod 1.16.2 , latest ruby 3.4.4 , ios 18.5 , simulator with device iphone 16 plus.

### Additional Information

*No response*

i have followed above step and no help , as soon as i run last step pod install --repo-update , i get below error:
Analyzing dependencies
Pre-downloading: `FirebaseFirestore` from ` URL tag `11.4.0`
cloud_functions: Using Firebase SDK version '11.4.0' defined in 'firebase_core'
firebase_auth: Using Firebase SDK version '11.4.0' defined in 'firebase_core'
firebase_messaging: Using Firebase SDK version '11.4.0' defined in 'firebase_core'
firebase_performance: Using Firebase SDK version '11.4.0' defined in 'firebase_core'
firebase_storage: Using Firebase SDK version '11.4.0' defined in 'firebase_core'
[!] Unable to find a specification for `firebase_core` depended upon by `cloud_functions`

You have either:
 * mistyped the name or version.
 * not added the source repo that hosts the Podspec to your Podfile.

----------------
I think you guys are adding wrong dependency version for cloud functions and firebase.

Could you kindly share the output of the `flutter doctor -v` command with us? That will help us better understand your environment and assist you more effectively.

sure here is output , i installed flutter via flutterflow desktop app only and hope android toolchain should not matter on mac os:
Last login: Wed Jun 18 13:16:22 on ttys000
diksha@Dikshas-MacBook-Air ~ % flutter doctor -v

┌─────────────────────────────────────────────────────────┐
│ A new version of Flutter is available! │
│ │
│ To update to the latest version, run "flutter upgrade". │
└─────────────────────────────────────────────────────────┘
[✓] Flutter (Channel stable, 3.27.3, on macOS 15.5 24F74 darwin-arm64, locale en-CA)
 • Flutter version 3.27.3 on channel stable at /Users/diksha/Library/Application Support/io.flutterflow.prod.mac/flutter
 • Upstream repository URL
 • Framework revision c519ee916e (5 months ago), 2025-01-21 10:32:23 -0800
 • Engine revision e672b006cb
 • Dart version 3.6.1
 • DevTools version 2.40.2

[✗] Android toolchain - develop for Android devices
 ✗ Unable to locate Android SDK.
 Install Android Studio from: URL
 On first launch it will assist you in installing the Android SDK components.
 (or visit URL for detailed instructions).
 If the Android SDK has been installed to a custom location, please use
 `flutter config --android-sdk` to update to that location.

[✓] Xcode - develop for iOS and macOS (Xcode 16.4)
 • Xcode at /Applications/Xcode.app/Contents/Developer
 • Build 16F6
 • CocoaPods version 1.16.2

[✓] Chrome - develop for the web
 • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[!] Android Studio (not installed)
 • Android Studio not found; download from URL
 (or visit URL for detailed instructions).

[✓] VS Code (version 1.101.0)
 • VS Code at /Applications/Visual Studio Code.app/Contents
 • Flutter extension version 3.112.0

[✓] Connected device (4 available)
 • iPhone 16 Plus (mobile) • A453322F-A3CB-459D-9565-B61135BA4762 • ios • com.apple.CoreSimulator.SimRuntime.iOS-18-5 (simulator)
 • macOS (desktop) • macos • darwin-arm64 • macOS 15.5 24F74 darwin-arm64
 • Mac Designed for iPad (desktop) • mac-designed-for-ipad • darwin • macOS 15.5 24F74 darwin-arm64
 • Chrome (web) • chrome • web-javascript • Google Chrome 137.0.7151.120

[✓] Network resources
 • All expected network resources are available.

! Doctor found issues in 2 categories.

**Hi there!**

Just a quick question—has the issue been resolved on your end with the latest FlutterFlow update?

If not, I recommend reaching out to our support team through the in-app chat or by emailing support@flutterflow.io. They’ll be able to provide more detailed guidance and assist you further, as this may be beyond what I can help with directly.

Thanks!

no , still same issue with latest 6.0.33 flutterflow desktop app

Hi , i reproduced this issue with a blank project , as soon as i add firebase project with basic auth and firebase storage this issue comes. If the project is without firebase then it works.

Hello!

Perfect—thanks for confirming! Could you please share the bug report code of the project here so we can take a closer look?

here is bug report code : IT40kvLlx5NgofVA0JDubPlagio5Jlw6a7gFiO1RaCosfZT2BLMcec/RVERvZ/bhYHx2eVmKo2oW09L3jYThGu5daUmsfYg5zLtqEzvhXkC5eLauFJWgT3NSO59RBUjG1pmziBF4D7J2SiQt1FyENdyQTXftGKbTClgsD6/LZO4=

I created new ticket as well for blank project failing which has more detail but similar only : URL

Here’s how you can fix the iOS pod install issue:

### 1\. Update CocoaPods Specs

In your project’s `ios/` folder, open a terminal and run:

cd ios
pod install --repo-update

This updates your local CocoaPods specs and installs dependencies in one step.

### 2\. Check Your Podfile

Make sure your `ios/Podfile` starts like this:

source ' URL
platform :ios, '12.0'

use_frameworks!
use_modular_headers!

target 'Runner' do
 flutter_install_all_ios_pods File.dirname(File.realpath(__FILE__))
end

### 3\. Clean Pods If the Issue Persists

If you’re still seeing errors, do a clean install:

cd ios
rm -rf Pods Podfile.lock
pod install --repo-update

### 4\. Rebuild the iOS App

From the root of your Flutter project:

flutter clean
flutter pub get
flutter run

### Why This Works

CocoaPods needs the latest podspecs to find newer versions like `firebase_core 11.4.0`. If they’re missing, pod install fails. Cleaning up Pods ensures no outdated configurations interfere.

Let us know if that helps or if you hit any other issues!

Again same repeated answers , i have taken all of the above steps multiple times and even beyond that.
Pls just try to simply add firebase integration in blank flutterflow project in the latest ios environment mentioned and u will see the issue , then try to resolve it.

Hello again! The reason I’m asking you to try that again is because the issue appears to be only reproducible on your end. Neither I nor any colleagues have been able to replicate it, even after trying the steps you mentioned.

Given that this seems to be project- or setup-specific rather than a general bug, I recommend reaching out to our Support team directly via support@flutterflow.io or through the in-app chat on the web platform. The GitHub issue tracker is intended for bugs that can be easily reproduced in a new, clean project, and Support is better equipped to handle more specific cases like this.

Thanks for your understanding!

Closing this issue since we haven't heard back from you. If you are still facing the problem please submit a new issue. Thank you!

Mac os local run fails as soon as firebase project added
### Can we access your project?

- [X] I give permission for members of the FlutterFlow team to access and test my project for the sole purpose of investigating this issue.

### Current Behavior

Macos local run fails with error : firebase_performance: Using Firebase SDK version '11.4.0' defined in 'firebase_core' \[!\] Unable to find a specification for `firebase_core` depended upon by `firebase_auth`. rememeber without firebase my project runs that means i dont have issues with environment on my system.

### Expected Behavior

This is empty project with just firebase added for auth and storage this is the most basic and empty setup. This is the most important function provided by flutterflow which is broken from at least 3 months. I have been raising tickets and flutterflow team closes it saying that they cant reproduce because my project had lot of things and they never tried to reproduce it with simple blank project.

the error log at very bottom shows garbage and confusing error related to cocoapod out of date but thats just confusing error , i have done every possible stuff present on internet and even idea provided by u guys to update it. I have created old tickets for same issue for my real project which were closed because it was big project and team said its specific to my project thats why i created now blank project and it fails with same issue : URL URL

### Steps to Reproduce

1. create basic project with firebase on latest macos

### Reproducible from Blank

- [X] The steps to reproduce above start from a blank project.

### Bug Report Code (Required)

IT40kvLlx5NgofVA0JDubPlagio5Jlw6a7gFiO1RaCosfZT2BLMcec/RVERvZ/bhYHx2eVmKo2oW09L3jYThGu5daUmsfYg5zLtqEzvhXkC5eLauFJWgT3NSO59RBUjG1pmziBF4D7J2SiQt1FyENdyQTXftGKbTClgsD6/LZO4=

### Visual documentation

### Environment

markdown
- FlutterFlow version: latest desktop APP
- Platform: MACOS
- Browser name and version:NA
- Operating system and version affected: latest M4 chip macbook , latest os : sequoia 15.5 , with latest xcode 16.4 , latest cocoapod 1.16.2 , latest ruby 3.4.4 , ios 18.5 , simulator with device iphone 16 plus.

### Additional Information

flutter doctor output :
i installed flutter via flutterflow desktop app only and hope android toolchain should not matter on mac os:
Last login: Wed Jun 18 13:16:22 on ttys000
diksha@Dikshas-MacBook-Air \~ % flutter doctor -v

┌─────────────────────────────────────────────────────────┐
│ A new version of Flutter is available! │
│ │
│ To update to the latest version, run "flutter upgrade". │
└─────────────────────────────────────────────────────────┘
\[✓\] Flutter (Channel stable, 3.27.3, on macOS 15.5 24F74 darwin-arm64, locale en-CA)
• Flutter version 3.27.3 on channel stable at /Users/diksha/Library/Application Support/io.flutterflow.prod.mac/flutter
• Upstream repository URL
• Framework revision c519ee916e (5 months ago), 2025-01-21 10:32:23 -0800
• Engine revision e672b006cb
• Dart version 3.6.1
• DevTools version 2.40.2

\[✗\] Android toolchain - develop for Android devices
✗ Unable to locate Android SDK.
Install Android Studio from: URL
On first launch it will assist you in installing the Android SDK components.
(or visit URL for detailed instructions).
If the Android SDK has been installed to a custom location, please use
flutter config --android-sdk to update to that location.

\[✓\] Xcode - develop for iOS and macOS (Xcode 16.4)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Build 16F6
• CocoaPods version 1.16.2

\[✓\] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

\[!\] Android Studio (not installed)
• Android Studio not found; download from URL
(or visit URL for detailed instructions).

\[✓\] VS Code (version 1.101.0)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.112.0

\[✓\] Connected device (4 available)
• iPhone 16 Plus (mobile) • A453322F-A3CB-459D-9565-B61135BA4762 • ios • com.apple.CoreSimulator.SimRuntime.iOS-18-5 (simulator)
• macOS (desktop) • macos • darwin-arm64 • macOS 15.5 24F74 darwin-arm64
• Mac Designed for iPad (desktop) • mac-designed-for-ipad • darwin • macOS 15.5 24F74 darwin-arm64
• Chrome (web) • chrome • web-javascript • Google Chrome 137.0.7151.120

\[✓\] Network resources
• All expected network resources are available.

! Doctor found issues in 2 categories.


Question: Standardising some of the synonym types?

@dosumis, regarding order of operations, do you recommend opening a ticket in OMO to incorporate Uberon's abbreviation AP and then pulling that into CL?

Standardising some of the synonym types
Typical ones we use are:

- layperson
- acronyms
- abbreviation

We may also need things like:

- machine-translated (google translate etc).

AFAIK, there is this problem with OBO format that it translated these:

synonym: "IGHD" EXACT ABBREVIATION [DOID:0060870]

Into:

This is very unfortunate, as it creates incompatible properties all across the OBO sphere.. What is least painful strategy to get out of this problem?

We are also keen on having abbreviation and acronym in CL and UBERON - see URL

Is a separate ticket required to bring the abbreviation and acronym synonym types into CL and UBERON? Kindly advise what is required to facilitate this. @shawntanzk, given your technical expertise, would you be able to help with the implementation plan?

lol 100% no expertise >.<

I think this requires community agreement and support/need to push this up the priority list more than anything

Create synonym_type_property 'abbreviation' for CL
UBERON has the Annotation property 'abbreviation' (Superproperty 'synonym_type_property'). URL

So editors can also leverage this property in CL, I request for Annotation property 'abbreviation' to be imported from UBERON into CL.

PR that inspired this request: PR #1440

would be a bit weird to add a uberon specific AP to CL. I've previously used exact synonym in CL but do understand that there might be use cases for that - thoughts on this @dosumis?

@shawntanzk (cc @bvarner-ebi )

While OBO members may not all agree on how strict we should be with the "exact" synonym scope, I do think that in CL there are plenty of use cases for the "abbreviation" annotation property. E.g. ESC, Treg, TAC... Abbreviations and acronyms are also discussed in the OBOOK here, so we wouldn't be going against consensus practice if we imported that AP. (Uberon has way more APs than CL and I think we've imported at least one before - location grouping - but I can't find mention of it.)

100% agree that abbreivation AP would be useful in CL, but I guess my question then is should we be adding a specific AP to CL? should this live in a more general location given its usefulness? If so where? This seems to vary and was wondering what the best way to handle this would be.

Some notes on this:
MONDO seems to have their own AP for abbreviation
Like things like hasexactsynonym has a GO iri ( URL

Yes, it seems that synonym policy/SOP aren't applied homogeneously throughout OBO. Understandably, reaching a consensus may not be a high priority. Would be great to have feedback from others. Perhaps @nicolevasilevsky , who wrote the Mondo synonyms guide, might comment please. :-)

Yes, we have our own AP in Mondo. HPO uses a similar AP too, that is their own as well: URL

I'd favour adding abbreviation as a synonym type to CL ASAP. Could re-use the Uberon one or add to CL following similar pattern. However, I think this should move to OMO or IAO ASAP. It will be a pain to query in integrated resources (e.g. Ubergraph) if every ontology has their own version.

Note - there may be some OBO format compatibility issues


Question: Some font will cause line break in -?

Titled "Bug in 3.141592653-2.6-0.999995 (TeX Live 2023) with fontspec and tabularray?"

Some font will cause line break in -
If using `xeCJK` and `\setmainfont{TeX Gyre Termes}`, `{-}` is right, but `-` will cause breakline. I found some in #257 and #145, but the situation is not quite the same.

latex
\documentclass{article}
\usepackage{xeCJK}
% \usepackage{fontspec}
\setmainfont{TeX Gyre Termes}
% \setmainfont{Latin Modern Roman}
\usepackage{tabularray}
\begin{document}
\begin{tblr}{colspec={l},hlines,vlines}
W-ADAPT\cite{test} \\
W{-}ADAPT\cite{test} \\
\end{tblr}
\begin{tabular}{|l|}
\hline
W-ADAPT\cite{test} \\
\hline
W{-}ADAPT\cite{test} \\
\hline
\end{tabular}
\end{document}

image

If I change `\setmainfont{TeX Gyre Termes}` to `\setmainfont{Latin Modern Roman}`. All is normal.

image

But if I change it to `\usepackage{fontspec}` and `\setmainfont{TeX Gyre Termes}` without using `xeCJK`.

image

It is a XeTeX bug (see #16), and LuaTeX produces correct result with the same example:

latex
\documentclass{article}
\usepackage{fontspec}
\setmainfont{TeX Gyre Termes}
\usepackage{tabularray}
\parindent=0pt
\begin{document}
\def\mytext{W-ADAPT[?]}
\setbox0=\hbox{\mytext}
\setbox2=\vbox{\hsize=\wd0 \mytext} \box2
\setbox2=\vbox{\hsize=\wd0 \raggedright \mytext} \box2
\setbox2=\vbox{\hsize=\wd0 \centering \mytext} \box2
\setbox2=\vbox{\hsize=\wd0 \raggedleft \mytext} \box2
\end{document}

But it seems the workaround in #16 doesn't work for this case.

Thank you for your answer, I have a certain understanding of it. But I am still curious why there are differences between different fonts? What factors of different fonts lead to this situation? I stumbled upon this issue by chance, and when I tried modifying some characters, the problem disappeared, which is really puzzling.

Sorry, I can't answer these questions. Only people familiar with XeTeX source code could answer them (and fix this bug).

I have found a solution in URL and I can deal with this problem again. Thank you very much for your answer.

A hack I came up with was to wrap cells in an `mbox`. For example this style specification

 row{1} = {font=\itshape, cmd=\mbox}

causes every cell on row 1 to be wrapped in an mbox, thus suppressing hyphen breaking for any hyphens on row 1. Alternatively, `cmd = \exhyphenpenalty10000` also works, at least until the bug is fixed.

Unwanted line break for some cell text
In fixing issue #15, I found another issue:

`latex
\documentclass[11pt]{article}
\usepackage{tabularray}
\begin{document}
\begin{tblr}{ll}
\hline
Src-Addr & Dest-Addr \\
\hline
\end{tblr}
\end{document}

`
image

The result is correct if we change font size to `10pt` or `12pt`.

It's a bug of XeTeX engine:
 URL
 URL
We don't need to fix it here.

With the latest package code, there is a wordaround for XeTeX users:

latex
\makeatletter
\AddToHook{cmd/TblrAlignBoth/after}{\@rightskip=-1sp \rightskip=\@rightskip}
\AddToHook{cmd/TblrAlignLeft/after}{\@rightskip=-1sp plus 1fil \rightskip=\@rightskip}
\AddToHook{cmd/TblrAlignCenter/after}{\@rightskip=-1sp plus 1fil \rightskip=\@rightskip}
\AddToHook{cmd/TblrAlignRight/after}{\leftskip=-1sp plus 1fil}
\makeatother

And it doesn't break any existing test files.

latex
\documentclass[11pt]{article}
\usepackage{tabularray}
\begin{document}

\begin{tblr}{jj}
\hline
Src-Addr & Dest-Addr \\
Src-Addr & Dest-Addr1111 \\
\hline
\end{tblr}

\begin{tblr}{ll}
\hline
Src-Addr & Dest-Addr \\
Src-Addr & Dest-Addr1111 \\
\hline
\end{tblr}

\begin{tblr}{cc}
\hline
Src-Addr & Dest-Addr \\
Src-Addr & Dest-Addr1111 \\
\hline
\end{tblr}

\begin{tblr}{rr}
\hline
Src-Addr & Dest-Addr \\
Src-Addr & Dest-Addr1111 \\
\hline
\end{tblr}

\bigskip
\makeatletter
\AddToHook{cmd/TblrAlignBoth/after}{\@rightskip=-1sp \rightskip=\@rightskip}
\AddToHook{cmd/TblrAlignLeft/after}{\@rightskip=-1sp plus 1fil \rightskip=\@rightskip}
\AddToHook{cmd/TblrAlignCenter/after}{\@rightskip=-1sp plus 1fil \rightskip=\@rightskip}
\AddToHook{cmd/TblrAlignRight/after}{\leftskip=-1sp plus 1fil}
\makeatother

\begin{tblr}{jj}
\hline
Src-Addr & Dest-Addr \\
Src-Addr & Dest-Addr1111 \\
\hline
\end{tblr}

\begin{tblr}{ll}
\hline
Src-Addr & Dest-Addr \\
Src-Addr & Dest-Addr1111 \\
\hline
\end{tblr}

\begin{tblr}{cc}
\hline
Src-Addr & Dest-Addr \\
Src-Addr & Dest-Addr1111 \\
\hline
\end{tblr}

\begin{tblr}{rr}
\hline
Src-Addr & Dest-Addr \\
Src-Addr & Dest-Addr1111 \\
\hline
\end{tblr}

\end{document}

image

There were some discussions on this XeTeX bug on tex-live mail list:

 URL

 URL


Question: test PPC with LLVM_IAS=1?

@MaskRay noted

git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
master and got a conflict.
Seems that URL
("[PATCH kernel v4] powerpc/64: Add UADDR64 relocation support") fixed
the issue.
It just doesn't change arch/powerpc/boot/crt0.S

So this seems implicitly fixed in -next.
commit d79976918852 ("powerpc/64: Add UADDR64 relocation support")

fedora and suse CI green runs.

test PPC with LLVM_IAS=1
@aik sent a series for enabling LLVM_IAS=1. @mpe accepted it, so it should be in linux-next soon. The cover letter mentions we need URL that @nemanjai is working on. As soon as that lands, we should be able to start testing probably our ppc64le configs on linux-next with clang-14 with LLVM_IAS=1 enabled.

It also needs the hwsync suport which is already merged in LLVM.

With D115419 applied and Linux at commit 112450df61b7 ("Merge branch 'i2c/for-mergewindow' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux"), we get closer to building a `powernv_defconfig` kernel with `LLVM=1`:

$ make -skj"$(nproc)" ARCH=powerpc LLVM=1 powernv_defconfig all
/usr/sbin/ld: unrecognised emulation mode: elf64lppc
Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu elf_l1om elf_k1om
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [arch/powerpc/kernel/vdso64/Makefile:44: arch/powerpc/kernel/vdso64/vdso64.so.dbg] Error 1
...

This is URL If I quickly hack around that error:

diff
diff --git a/arch/powerpc/kernel/vdso64/Makefile b/arch/powerpc/kernel/vdso64/Makefile
index 3c5baaa6f1e7..45dc62ba1541 100644
--- a/arch/powerpc/kernel/vdso64/Makefile
+++ b/arch/powerpc/kernel/vdso64/Makefile
@@ -39,6 +39,8 @@ asflags-y := -D__VDSO64__ -s
 targets += vdso64.lds
 CPPFLAGS_vdso64.lds += -P -C -U$(ARCH)

+vdsold-$(CONFIG_LD_IS_LLD) := -fuse-ld=lld
+
 # link rule for the .so file, .lds has to be first
 $(obj)/vdso64.so.dbg: $(src)/vdso64.lds $(obj-vdso64) $(obj)/vgettimeofday.o FORCE
 $(call if_changed,vdso64ld_and_check)
@@ -53,4 +55,4 @@ include/generated/vdso64-offsets.h: $(obj)/vdso64.so.dbg FORCE

 # actual build commands
 quiet_cmd_vdso64ld_and_check = VDSO64L $@
- cmd_vdso64ld_and_check = $(CC) $(c_flags) -o $@ -Wl,-T$(filter %.lds,$^) $(filter %.o,$^); $(cmd_vdso_check)
+ cmd_vdso64ld_and_check = $(CC) $(c_flags) $(vdsold-y) -o $@ -Wl,-T$(filter %.lds,$^) $(filter %.o,$^); $(cmd_vdso_check)

I get another one:

ld.lld: warning: cannot find entry symbol _start; not setting start address
objcopy: Unable to recognise the format of the input file `vmlinux'
make[2]: *** [arch/powerpc/boot/Makefile:391: arch/powerpc/boot/zImage.pseries] Error 1
objcopy: Unable to recognise the format of the input file `vmlinux'
make[2]: *** [arch/powerpc/boot/Makefile:391: arch/powerpc/boot/zImage.epapr] Error 1
make[2]: Target 'arch/powerpc/boot/zImage' not remade because of errors.
make[1]: *** [arch/powerpc/Makefile:283: zImage] Error 2

Looks like `arch/powerpc/boot/wrapper` has hardcoded references to the GNU tools (e.g. `${CROSS}objcopy`), so we cannot omit `CROSS_COMPILE` for now. After adding `CROSS_COMPILE=powerpc64le-linux-gnu-` back to the command line, I get a kernel that boots in QEMU:

$ boot-qemu.sh -a ppc64le -k . -t 30s
QEMU location: /qemu/bin

QEMU version: QEMU emulator version 6.2.50 (v6.2.0-39-g0da6106883)
...
[ 0.000000][ T0] Linux version 5.16.0-09670-g112450df61b7-dirty (nathan@archlinux-ax161) (ClangBuiltLinux clang version 14.0.0 ( URL 1344e35c9822db4998d9eec7f219ecab700ecaa1), LLD 14.0.0) #1 SMP Fri Jan 14 17:53:20 UTC 2022
...

`ppc64le_defconfig` builds as well.

Very close!

When I looked last time I explored this for hermetic builds a couple of years ago, it was not super apparent what the fix should be because it sounds like `arch/powerpc/boot/wrapper` is used in other projects so we cannot just convert those references over to `${OBJCOPY}` or the like (@mpe can correct me if I am wrong). It looks like @gwelymernans touched this area in 5.11, perhaps he could shed some light on how we might work around this.

Do you have a link to the discussion from a couple of years ago?

AFAIK the wrapper is not used in any other projects, although that was the idea of it originally.

It is potentially called by other scripts from outside the kernel tree. I think in the old days maybe some distro scripts called it, but I think the modern distros don't. Possibly some 32-bit embedded stuff still uses it.

Also eg. see torvalds/linux@528229d210781b2da66c6d257a326c21099982b5, which talks about "the signing tool" calling the wrapper script. I'm not sure that signing tool was ever released, and it's basically dead now, but that's the kind of thing we have to keep in mind, external scripts that call the wrapper and expect it to behave a certain way.

So the main thing is to make any change backward compatible, if possible. ie. so that an existing caller will get the same behaviour it used to.

What is the LLVM alternative to `objcopy` ?

I have tried to find it but I do not recall ever bringing this up to upstream, so I could be misremembering something... _shrugs_

Ack. I have hacked something up that works for `powernv_defconfig` when building `zImage.epapr`:

 URL

$ make -skj"$(nproc)" ARCH=powerpc LLVM=1 distclean powernv_defconfig zImage.epapr
ld.lld: warning: cannot find entry symbol _start; not setting start address

$ boot-qemu.sh -a ppc64le -k . -t 30s
QEMU location: /usr/bin

QEMU version: QEMU emulator version 6.2.0
...
[ 0.000000][ T0] Linux version 5.16.0-11415-gc1a97611aaa2 (nathan@dev-arch.archlinux-ax161) (ClangBuiltLinux clang version 14.0.0 ( URL d4ffb6ec65d7173ee63fa0e70eedd049944ceeff), LLD 14.0.0) #1 SMP Fri Jan 21 12:16:52 MST 2022
...

However, the build fails when building `zImage.pseries` (included in the `all` target, obviously related to the new use of `ld.lld`):

$ make -skj"$(nproc)" ARCH=powerpc LLVM=1 distclean powernv_defconfig all
...
ld.lld: error: relocation R_PPC64_ADDR64 cannot be used against symbol '_start'; recompile with -fPIC

ld.lld: error: relocation R_PPC64_ADDR64 cannot be used against symbol '_etext'; recompile with -fPIC

ld.lld: error: relocation R_PPC64_ADDR64 cannot be used against symbol '__bss_start'; recompile with -fPIC

ld.lld: error: relocation R_PPC64_ADDR64 cannot be used against symbol '_end'; recompile with -fPIC

make[2]: *** [arch/powerpc/boot/Makefile:400: arch/powerpc/boot/zImage.pseries] Error 1
...

If I apply the same fix as commit 4c078c86b4a4 ("powerpc/boot/wrapper: Add "-z notext" flag to disable diagnostic") to the `pseries` case in `arch/powerpc/boot/wrapper`, `zImage.pseries` builds successfully and I can boot it in QEMU, just like `zImage.epapr`.

There was one other issue I noticed with `ppc44x_defconfig` (the 32-bit config that we test and boot) related to `relro`. I solved this by adding `-z norelro`.

`llvm-objcopy` :)

I realized that `tuxsuite` provides `CROSS_COMPILE` so we can actually enable this configuration with `LLVM_IAS=1` as soon as URL is merged, which will allow us to workaround URL and URL

I am reopening this due to #324 and URL I build tested it locally but did not boot test.

I will likely just build these configurations with `ld.bfd` in the meantime but I am open to other suggestions.

So we have a few different ways to work around URL

1. Disable the boot tests for the distribution configurations.
2. Disable `CONFIG_RELOCATABLE=y` for the distribution configurations.
3. Use `ld.bfd` instead of `ld.lld`.

@nickdesaulniers do you have a preference?

IMHO `ld.lld` having an "imprecise" value for `DT_RELACOUNT` is just plainly a bug in `ld.lld`.

Our relocation code is primitive, it runs very very early in boot, and no doubt it could be improved, but it has worked for the last ~13 years with no issue.

Boot hang with certain PowerPC configurations after da0e5b885b25cf4ded0fa89b965dc6979ac02ca9
After URL both Fedora and OpenSUSE's powerpc64le configurations fail to boot for me in QEMU. On v5.17-rc3:

$ curl -LSso .config URL

$ make -skj"$(nproc)" ARCH=powerpc CROSS_COMPILE=powerpc64le-linux-gnu- LLVM=1 LLVM_IAS=0 olddefconfig zImage.epapr

$ boot-qemu.sh -a ppc64le -k . -t 45s
...
[ 10.042804807,5] INIT: Waiting for kernel...
[ 10.042963994,5] INIT: platform wait for kernel load failed
[ 10.043178236,5] INIT: Assuming kernel at 0x20000000
[ 10.043400737,5] INIT: 64-bit LE kernel discovered
[ 10.043816082,2] NVRAM: Failed to load
[ 10.092415012,5] INIT: Starting kernel at 0x20010000, fdt at 0x3068dbd0 16441 bytes

zImage starting: loaded at 0x0000000020010000 (sp: 0x0000000020cdcda0)
Allocating 0x2c37501 bytes for kernel...
Decompressing (0x0000000000000000 <- 0x0000000020023000:0x0000000020cdb275)...
Done! Decompressed 0x2a60000 bytes

Linux/PowerPC load:
Finalizing device tree... flat tree at 0x20cddca0
qemu-system-ppc64: terminating on signal 15 from pid 863670 (timeout)
+ RET=124
+ set +x

At the commit prior to that one, the kernel fires right up:

[ 10.046432374,5] INIT: Waiting for kernel...
[ 10.046600313,5] INIT: platform wait for kernel load failed
[ 10.046828764,5] INIT: Assuming kernel at 0x20000000
[ 10.047069394,5] INIT: 64-bit LE kernel discovered
[ 10.047512051,2] NVRAM: Failed to load
[ 10.097703665,5] INIT: Starting kernel at 0x20010000, fdt at 0x3068dbd0 16441 bytes

zImage starting: loaded at 0x0000000020010000 (sp: 0x0000000020cdcda0)
Allocating 0x2c37501 bytes for kernel...
Decompressing (0x0000000000000000 <- 0x0000000020023000:0x0000000020cdb254)...
Done! Decompressed 0x2a60000 bytes

Linux/PowerPC load:
Finalizing device tree... flat tree at 0x20cddca0
[ 0.000000] dt-cpu-ftrs: setup for ISA 2070
[ 0.000000] dt-cpu-ftrs: not enabling: subcore (unknown and unsupported by kernel)
[ 0.000000] dt-cpu-ftrs: final cpu/mmu features = 0x000000fb8f5db187 0x3c006001
[ 0.000000] hash-mmu: Page sizes from device-tree:
[ 0.000000] hash-mmu: base_shift=12: shift=12, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=0
[ 0.000000] hash-mmu: base_shift=12: shift=16, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=7
[ 0.000000] hash-mmu: base_shift=12: shift=24, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=56
[ 0.000000] hash-mmu: base_shift=16: shift=16, sllp=0x0110, avpnm=0x00000000, tlbiel=1, penc=1
[ 0.000000] hash-mmu: base_shift=16: shift=24, sllp=0x0110, avpnm=0x00000000, tlbiel=1, penc=8
[ 0.000000] hash-mmu: base_shift=24: shift=24, sllp=0x0100, avpnm=0x00000001, tlbiel=0, penc=0
[ 0.000000] hash-mmu: base_shift=34: shift=34, sllp=0x0120, avpnm=0x000007ff, tlbiel=0, penc=3
[ 0.000000] Enabling pkeys with max key count 32
[ 0.000000] Activating Kernel Userspace Access Prevention
[ 0.000000] Activating Kernel Userspace Execution Prevention
[ 0.000000] Using 1TB segments
[ 0.000000] hash-mmu: Initializing hash mmu with SLB
[ 0.000000] Linux version 5.17.0-rc3 (nathan@dev-arch.archlinux-ax161) (ClangBuiltLinux clang version 14.0.0 ( URL 460830a9c664e8cce959c660648faa7747ad8bdc), LLD 14.0.0) #1 SMP Mon Feb 7 16:59:00 MST 2022
[ 0.000000] Found initrd at 0xc000000028000000:0xc000000028856200
...

I have done no further triage, as I am currently testing a few different things. cc @MaskRay in case something obvious sticks out to you. I am happy to provide any information that might make figuring out what is going wrong any easier.

I encounter a pahole (my `apt info dwarves` is `1.20-1+gl0`) issue.

 LD .tmp_vmlinux.btf
 BTF .btf.vmlinux.bin.o
[750483] INT DW_ATE_unsigned_56 Error emitting BTF type
Encountered error while encoding BTF.
 LD .tmp_vmlinux.kallsyms1
 KSYMS .tmp_vmlinux.kallsyms1.S
 AS .tmp_vmlinux.kallsyms1.S
 LD .tmp_vmlinux.kallsyms2
 KSYMS .tmp_vmlinux.kallsyms2.S
 AS .tmp_vmlinux.kallsyms2.S
 LD vmlinux
 BTFIDS vmlinux
FAILED: load BTF from vmlinux: Invalid argument
make[2]: *** [/home/maskray/Dev/linux/Makefile:1155: vmlinux] Error 255
make[2]: *** Deleting file 'vmlinux'
make[1]: *** [/home/maskray/Dev/linux/Makefile:350: __build_one_by_one] Error 2
make[1]: Leaving directory '/tmp/linux/ppc64le'
make: *** [Makefile:219: __sub-make] Error 2

The change refactored how `DT_RELACOUNT` (shown as `RELACOUNT` in `readelf -d` output), which is defined for `-z combreloc` (default): the number of `R_*_RELATIVE` relocations in the beginning of `.rela.dyn`. I can find some `RELACOUNT` usage in arch/powerpc/ . Is the `DT_RELACOUNT` value now wrong?

% rg RELACOUNT
include/uapi/linux/elf.h
108:#define DT_RELACOUNT 0x6ffffff9

arch/powerpc/kernel/reloc_64.S
11:RELACOUNT = 0x6ffffff9
30: * Scan the dynamic section for the RELA and RELACOUNT entries.
41:2: addis r6,r6,(-RELACOUNT)@ha
42: cmpdi r6,RELACOUNT@l
44: ld r8,8(r11) /* get RELACOUNT value in r8 */
47:4: cmpdi r7,0 /* check we have both RELA and RELACOUNT */

arch/powerpc/boot/crt0.S
11:RELACOUNT = 0x6ffffff9
68: * We need the RELA and RELACOUNT entries. */
78:11: addis r8,r8,(-RELACOUNT)@ha
79: cmpwi r8,RELACOUNT@l
81: lwz r0,4(r12) /* get RELACOUNT value in r0 */
88: * The R_PPC_RELATIVE ones come first and there are RELACOUNT
163:10: addis r12,r12,(-RELACOUNT)@ha
164: cmpdi r12,RELACOUNT@l
166: ld r8,8(r11) /* get RELACOUNT value in r8 */
170: cmpdi r13,0 /* check we have both RELA and RELACOUNT */

`vmlinux` has some position-independent long branch thunks which need `R_*_RELATIVE` relocations. In such a case, ld.lld generated `DT_RELACOUNT` value is smaller than the actual value. I can make it correct but.. well, it's just more code without many benefits.

glibc logic is the following and a smaller DT_LACOUNT value will still work.

c
Treat the DT_RELACOUNT relocations as R_PPC64_RELATIVE. Don't check their type

For the rest relocations, check the type and perform R_PPC64_RELATIVE/R_PPC64_ADDR64/...

The arch/powerpc code just seems to assume that there is no `R_PPC64_RELATIVE` after the first `DT_RELACOUNT` relocations.

There's probably some important context as _why_ the number of that type of relocation is hard coded in the source...

It appears that `arch/powerpc/kernel/reloc_64.S` is the code that handles these relocations. I am not sure how it should be reworked to handle `ld.lld`'s imprecise `DT_RELACOUNT` value.

cc @aik since I noticed you opened URL above.

This patch can fix the issue

patch
From 9822ba2edb1808a2cf3a4690178d20a0ea8dfd36 Mon Sep 17 00:00:00 2001
From: Fangrui Song
Date: Tue, 8 Mar 2022 21:31:51 -0800
Subject: [PATCH] ppc64: Replace DT_RELACOUNT usage with DT_RELASZ

DT_RELACOUNT is an ELF dynamic tag inherited from SunOS indicating the number
of R_*_RELATIVE relocations. It is optional but {ld.lld,ld.lld} -z combreloc
always creates it (if non-zero) to slightly speed up glibc ld.so relocation
resolving. The tag is otherwise nearly unused.

lld>=15 (since commit da0e5b885b25cf4ded0fa89b965dc6979ac02ca9) underestimates
DT_RELACOUNT for ppc64 when position-independent long branch thunks are used.
Since our code always compares the relocation type with R_PPC64_RELATIVE,
replacing every occurrence of DT_RELACOUNT with
DT_RELASZ/sizeof(Elf64_Rela)=DT_RELASZ/24 is correct. Make such changes to
avoid using the obscure DT_RELACOUNT.

For an unsigned 32-bit integer (the bound of DT_RELASZ), x divided by 24
can be implemented as x*0xaaaaaaab >> 4.

Link: URL
Reported-by: Nathan Chancellor
Signed-off-by: Fangrui Song
---
 arch/powerpc/boot/crt0.S | 28 +++++++++++++++++-----------
 arch/powerpc/kernel/reloc_64.S | 15 +++++++++------
 2 files changed, 26 insertions(+), 17 deletions(-)

diff --git a/arch/powerpc/boot/crt0.S b/arch/powerpc/boot/crt0.S
index feadee18e271..1c96ebe7ef1a 100644
--- a/arch/powerpc/boot/crt0.S
+++ b/arch/powerpc/boot/crt0.S
@@ -8,7 +8,7 @@
 #include "ppc_asm.h"

 RELA = 7
-RELACOUNT = 0x6ffffff9
+RELASZ = 8

 .data
 /* A procedure descriptor used when booting this as a COFF file.
@@ -65,7 +65,7 @@ p_base: mflr r10 /* r10 now points to runtime addr of p_base */
 subf r11,r11,r12 /* runtime - linktime offset */

 /* The dynamic section contains a series of tagged entries.
- * We need the RELA and RELACOUNT entries. */
+ * We need the RELA and RELASZ entries. */
 li r9,0
 li r0,0
 9: lwz r8,0(r12) /* get tag */
@@ -75,18 +75,21 @@ p_base: mflr r10 /* r10 now points to runtime addr of p_base */
 bne 11f
 lwz r9,4(r12) /* get RELA pointer in r9 */
 b 12f
-11: addis r8,r8,(-RELACOUNT)@ha
- cmpwi r8,RELACOUNT@l
+11: cmpwi r8,RELASZ
 bne 12f
- lwz r0,4(r12) /* get RELACOUNT value in r0 */
+ lwz r0,4(r12) /* get RELASZ / 24 in r0 */
+ lis r8,0xaaaa
+ ori r8,r8,0xaaab
+ mulhwu r0,r0,r8
+ srwi r0,r0,4
 12: addi r12,r12,8
 b 9b

 /* The relocation section contains a list of relocations.
 * We now do the R_PPC_RELATIVE ones, which point to words
 * which need to be initialized with addend + offset.
- * The R_PPC_RELATIVE ones come first and there are RELACOUNT
- * of them. */
+ * The R_PPC_RELATIVE ones come first and there are at most
+ * RELASZ/24 of them. */
 10: /* skip relocation if we don't have both */
 cmpwi r0,0
 beq 3f
@@ -160,14 +163,17 @@ p_base: mflr r10 /* r10 now points to runtime addr of p_base */
 bne 10f
 ld r13,8(r11) /* get RELA pointer in r13 */
 b 11f
-10: addis r12,r12,(-RELACOUNT)@ha
- cmpdi r12,RELACOUNT@l
+10: cmpdi r12,RELASZ
 bne 11f
- ld r8,8(r11) /* get RELACOUNT value in r8 */
+ ld r8,8(r11) /* get RELASZ / 24 in r8 */
+ lis r0,0xaaaa
+ ori r0,r0,0xaaab
+ mulhwu r8,r8,r0
+ srwi r8,r8,4
 11: addi r11,r11,16
 b 9b
 12:
- cmpdi r13,0 /* check we have both RELA and RELACOUNT */
+ cmpdi r13,0 /* check we have both RELA and RELASZ */
 cmpdi cr1,r8,0
 beq 3f
 beq cr1,3f
diff --git a/arch/powerpc/kernel/reloc_64.S b/arch/powerpc/kernel/reloc_64.S
index 02d4719bf43a..362be759609f 100644
--- a/arch/powerpc/kernel/reloc_64.S
+++ b/arch/powerpc/kernel/reloc_64.S
@@ -8,7 +8,7 @@
 #include

 RELA = 7
-RELACOUNT = 0x6ffffff9
+RELASZ = 8
 R_PPC64_RELATIVE = 22

 /*
@@ -27,7 +27,7 @@ _GLOBAL(relocate)
 add r10,r10,r12 /* r10 has runtime addr of _stext */

 /*
- * Scan the dynamic section for the RELA and RELACOUNT entries.
+ * Scan the dynamic section for the RELA and RELASZ entries.
 */
 li r7,0
 li r8,0
@@ -38,13 +38,16 @@ _GLOBAL(relocate)
 bne 2f
 ld r7,8(r11) /* get RELA pointer in r7 */
 b 3f
-2: addis r6,r6,(-RELACOUNT)@ha
- cmpdi r6,RELACOUNT@l
+2: cmpdi r6,RELASZ
 bne 3f
- ld r8,8(r11) /* get RELACOUNT value in r8 */
+ ld r8,8(r11) /* get RELA / 24 in r8 */
+ lis r0,0xaaaa
+ ori r0,r0,0xaaab
+ mulhwu r8,r8,r0
+ srwi r8,r8,4
 3: addi r11,r11,16
 b 1b
-4: cmpdi r7,0 /* check we have both RELA and RELACOUNT */
+4: cmpdi r7,0 /* check we have both RELA and RELASZ */
 cmpdi cr1,r8,0
 beq 6f
 beq cr1,6f
--
2.35.1

Thanks a lot Fangrui! That patch works for me, I have provided a tag upstream.

 URL

cc @aik @mpe on @MaskRay 's comment about arch/powerpc/boot/crt0.S. Should @MaskRay resend just his hunks against that patch?

Do note that both of those runs did not use `LD=ld.lld` (see the `CC=clang`). Let me test `next-20220310` with both of those configurations locally then I will update URL by dropping Fangrui's patch and applying Alexey's patch to mainline.

Merged into mainline: URL

I am not sure this would be an acceptable backport for stable, so I am just going to close this up for now.

As d799769188529abc6cbf035a10087a51f7832b6b also resolves #1594, I have started a thread about whether or not applying that change to stable is appropriate.


Question: 2-way sync maybe possible now?

Hi, unfortunately it only updates MAL after you have actively watched something.
That being said I will add this to the main project as a possible addition later on down the line, as it seems to be a good feature to add to the plugin.

2-way sync maybe possible now?
With the higher accuracy, maybe a 2-way sync like the trakt plugin is now possible?

Maybe there is a somewhat reasonable way to-do this now, its just super convienent to have all your watched status in trakt and simply do a sync on a fresh install, so everything is back where you left.

So for normal shows/movies using the trakt plugin i dont have to care about any backup or library state, i can just reinstall the server point it to my libs and via a trakt resync all watched states are set to where i left off.

Would be super cool to have this also for Anime, but i understand that this can be more complex to get right.

So instead of doing an "outwards" sync (Jellyfin -> AniList/MAL/Kitsu), we sync the Jellyfin library to a provider (AniList/MAL/Kitsu -> Jellyfin)?
This has been requested before here #12.
While it wasn't exactly the original idea of the plugin, I still feel like it falls into the same realm.
We could even make it an optional scheduled event, for those who update their AniList/MAL/Kitsu manually outside of Jellyfin.
I can't see how it would be that difficult to achieve, simply retrieve the users complete/in progress list and update the Jellyfin library.
I'll look into it and keep you updated.

The main issue is you need to-be very carefully how/what you sync and also offer/study the settings options the trakt plugin has.

There can be a catastrophic case, where you wrongly sync the "unwatched" state to from remote to the local library and than quickly via a scheduled task, resync this new bad "unwatched" state back to the external list, basically wiping your watched states.

To prevent this case all trakt plugins by default disable remote->local "unwatched" sync via settings and also arrange the update jobs, so that the user has a chance to fix setting issues, before a full resync happens.

So by default you want the plugin to handle mainly "watched" states between local remote lists and via settings allow certain "unwatched" state combinations.

As example, the trakt plugin has specific settings to only set remote "watched/unwatched" if the user manually set the state. This is safer, since we detect a specific user toggled state change instead of doing some general assumptions.

So if you don't use the trakt plugin already, i strongly recommend to check it out and what settings combinations it offers, so you can avoid running into those catastrophic cases. We want safe defaults and carefully worded settings, similar to trakt so users quickly understand whats happening.

@vosmiic btw i took the liberty to add the plugin to the official documentation: URL

Ah great, thanks. I think we are now in a comfortable enough position to have it there.

Just as a FYI, #12 isn't really about what @Andy2244 is asking for here, which is importing watch status from a remote Trakt-like third-party provider (i.e. AniList/MAL/Kitsu -> Jellyfin) but rather an extended functionality of what Ani-Sync already does, which is exporting watch status to a third-party provider (i.e. Jellyfin -> AniList/MAL/Kitsu) but _without_ the hard requirement that I must first initiate playback of an anime for AniSync to scan it and update its remote status.

My issue (and I'm pretty certain I'm far from the only one affected) is that many of my anime were already watched or in various states of _media res_ before I ever installed AniSync, but I'd just never bothered to manually update my MAL profile. So what I'd love for AniSync to be able to do, and what I asked for in #12, would be that it be able to scan my local Jellyfin library, find which anime I have already watched and which shows I'm currently watching, and then update my MAL profile accordingly. And if it could also add my completely untouched anime to the "plan to watch" category, that would be even more awesome.

After this has been done, then adding what @Andy2244 is actually asking for, i.e. the ability to also import watch state from MAL et al to Jellyfin, would be the cherry on top.

I kinda included your scenario, its just a subpart of what a full 2-way sync does.

I know, I was replying to @vosmiic here:

"This" isn't what has been requested in #12 :)

Yeah sorry I got a bit confused.
I think I understand what would like to be achieved here, as it has been said before we would like both way sync between Jellyfin and the providers.
I also understand what @Andy2244 said with being careful how we handle it. Need to make sure it doesn't turn into a nasty loop of updating each others lists and end up with broken watch statuses on both sides. I can see that you explained how the trakt plugin handles it, and it seems to be the best method.
I'll take all of this into account when implementing these changes.

Hitting into some trouble syncing Jellyfin from the providers (AniList/MAL/Kitsu -> Jellyfin).
As most of the APIs don't contain the season number, we can't actually tell what season the user is on. Luckily each item in the providers database is a separate season.
The obvious way to accomplish this is to get the users watch list from the appropriate provider, get the IDs of the anime in said list and then run them through the anime offline database to get the AniDB ID and AniList ID of each season (if we are not using it as the selected provider to sync from) and then cross reference that to Jellyfin's database to see which seasons we have locally that match the IDs (we cannot use the local anime list XML file as that would only give us a TVDB ID which is bound to the series, not the season).
Only problem being is that Jellyfin does not seem to store season provider IDs, at least not for me. It stores the _series_ provider IDs, but not separate seasons.
The only time that we will get a match with a provider ID from the Jellyfin database is if we are working with the first season of a show from the users watch list.
Could someone please confirm that it isn't only me not having any provider IDs (labelled "external IDs" in the Jellyfin interface) for seasons? I have tried running the metadata fetcher multiple times and I don't get any extra provider IDs for seasons. I get a TVDB ID, but when I cross reference it with the XML file I get no results. I'm not entirely sure what the ID represents as I even tried looking it up on TVDB but got no results.

I have now got it updating shows from a users AniList watched list.
Just need to clean it up and get it working with other providers.

Forgot to update here recently but I believe I have got updating the Jellyfin library from providers anime list fully working.
I have added a warning to let users know that the feature is in "alpha" and that it may disrupt their Jellyfin libraries.
Next to do is to get the other direction working (Jellyfin to provider anime list).

This feature is now fully implemented on master.
Just doing some testing and then I should be able to release a new version with this functionality included.

Okay, the update with this has been released. Currently testing it locally and I am not hitting into any trouble.
That being said, I would be very careful using this - it could quite easily mess up your Jellyfin library or provider watch lists.

Update playback status for previously watched items
Just a quick question, will this scan for already watched anime and update their playback status on MAL, or will it only work for anime that I watch from now on? That would be a real bummer, because most of my stuff has already been watched back when this nice plugin wasn't yet around... :)


Question: INC0112560 - NR 3606562 showing as rejected and also "refund requested"?

From @yuisotozaki :

Here's the preview of the messages:

Scenario 1 - When a user attempts to edit or request refund, if the following conditions are met:

a. the request is in examination or has been examined but the screen the user is viewing does not yet reflect this state,

b. OR the request is being editing by another user.

Message: Please refresh the page or try again later, as there may be simultaneous updates in progress. This can happen if a name examiner is examining this name request. This can also happen if there is another user making changes to this name request or there is another tab on your browser with this name request in edit mode.

Scenario 2 - When a user is about to confirm the NR content and go to payment, to avoid duplicate request, an error shows up when the following conditions are met:

a. there exists an NR requested by the same user,

b. AND all the names exact match the names of the current request,

c. AND in pending for review or payment pending status.

Message: You have already created an identical name request. To request a different name, please go back and change the name to request and try again.

INC0112560 - NR 3606562 showing as rejected and also "refund requested"
### Description

Got this report from Debbie today, Oct 6th (**INC0112560**). Don't think there is any ops work, but created a bug to prevent this from occurring again.

It seems that this NR was able to be cancelled even though it was rejected first.
OR
It was rejected, even though it was cancelled first?
I can't tell from the transaction history.

Also in the transaction history, it says that when the client went to edit their NR, the entity type is "N/A". That is kinda weird, is that related?

### Reproduce
Try rejecting a cancelled name
Also try "cancel and refund"ing a rejected name

### Expected behavior
* Clients **can't** see the "Cancel and refund" button, or use the feature, if the NR has already been approved OR rejected

* Staff **Can't** reject or approve an NR that has been "cancel and refund"ed

### Impact
Bad data, incorrect actions being taken, breaking business rules

### Chance of occurring
Unsure

@pwei1018 @PCC199 there are two active ops tickets related to this investigation, can we please reopen for grooming.

 URL
 URL

According to the events record, the NR checked in right before refund. It is good:
image.png

So, the problem come from checkin/checkout process. Should the name states back to previous, the same as request state, when checkin?

Probably is about the conflict between namerequst and name-examination.
The **[Edit]** button in **Name Request** should not be available when the name state is changed to INPROCESS in Name-Examination.
So do the **[Upgrade Priority]** and **[Cancel and Refund]** buttons in **Name Request**.

The changes are apply to namerequest. Hold it for "Way Of Navigating" to be done.

@yuisotozaki @oanyahuru
Seeking UX assistance to improve the phrasing in the Error Dialog:
Current error dialog is like:
Private Zenhub Image

I suggest replacing the highlighted text with something along the lines of: 'The Name Request is currently under examination and editing/canceling is not available. For assistance, please contact BC Registries staff:

I would go further and update the title for this scenario.

Title: "This Name Request is being examined by BC Registries staff"
Description: "This Name Request is currently under examination. It cannot be edited or cancelled until the examination is complete. For assistance, please contact BC Registries staff:"

If we know more about how long a typical examination takes, we can include that in the message so that the user knows how long to wait before checking on the NR's status. For example, if name examination + system synch time is around 30 minutes, we can use the following message.

Description: "This Name Request is currently under examination. It cannot be edited or cancelled until the examination is complete. The approval status will reflect the result of the examination. This process typically takes about 30 minutes. For assistance, please contact BC Registries staff:"

I don't think the previous word in Error dialog (for payment) works for the scenario. After discuss with Yui, I opened URL for UX to review and decide the error message.
The modifications in error.vue may be temporary, with the hope that they will suffice until UX provides further guidance.

@eve-git I tested following the steps you outlined earlier (in your comment on June 28th). The 'edit' and the 'cancel and refund' buttons did not work once the NR status changed to "In Progress" on NameX. it rather displayed an error.

Private Zenhub Image

When I refreshed the page on Name Request, only the download receipts button was displayed when the request status was 'in progress'. As expected, clicking on it produced another error message

Private Zenhub Image

The error for the receipt shouldn't be there. I will look into it.

Looks like the receipt error is from pay-db and not in the scope of this ticket.

2023-09-27 17:39:11,309 - namex - DEBUG in payments:payments.py:13 - get_payment: {'_links': {'collection': '/api/v1/payment-requests?invoice_id=713929', 'self': '/api/v1/payment-requests/713929'}, 'businessIdentifier': 'NR 5795820', 'corpTypeCode': 'NRO', 'createdBy': 'IDIR\\OANYAHUR', 'createdName': 'Oge Anyahuru', 'createdOn': '2023-09-27T17:03:59+00:00', 'details': [{'label': 'NR Number:', 'value': 'NR 5795820'}, {'label': 'Name Choices:', 'value': ''}, {'label': '1.', 'value': 'RIGSHORE AND HORSE POWER TOOLS LTD.'}], 'id': 713929, 'isPaymentActionRequired': False, 'lineItems': [{'description': 'Name Request fee', 'filingFees': 30.0, 'futureEffectiveFees': 0.0, 'gst': 0.0, 'id': 716283, 'priorityFees': 0.0, 'pst': 0.0, 'quantity': 1, 'serviceFees': 0.0, 'statusCode': 'ACTIVE', 'total': 30.0, 'waivedBy': None, 'waivedFees': 0.0}], 'paid': 0.0, 'paymentAccount': {'accountId': '1999', 'accountName': "Ministry of Citizens' Services-BC Registries and Online Services Staff", 'billable': True}, 'paymentMethod': 'INTERNAL', 'refund': 0.0, 'routingSlip': '206462426', 'serviceFees': 0.0, 'statusCode': 'APPROVED', 'total': 30.0}
2023-09-27 17:39:11,309 - namex - DEBUG in payment:payment.py:298 - get: adding temporary receipt details.
Error when calling call_api get ( URL
2023-09-27 17:39:12,767 - namex - ERROR in api_resource:api_resource.py:14 - handle_exception: Error: SBCPaymentException('SBC Pay API exception.')
Error when calling call_api get ( URL
2023-09-27 17:40:42,229 - namex - ERROR in api_resource:api_resource.py:14 - handle_exception: Error: SBCPaymentException('SBC Pay API exception.')

@eve-git, if everything else is ok, I will move the ticket to Ready for Release. Another ticket can be opened for the receipt error.

Name Request: Enhance the error message in the Error Dialog to provide specificity and better meaning
Currently the error messages in Error Dialog are exclusively related to payment failures . This can cause confusion and make it challenging to tell what exact the error is:

Private Zenhub Image

The error messages should be reevaluated in the following scenarios:
1. URL when a request to be edited/refunded, and an error raised by the below reasons :
 a. the request is in examination or exanimated by a staff
 b. OR the request is editing by another user.

2. URL when a request is about to be confirmed and go to payment. To avoid duplicate request, an error shows up when the following conditions satisfied:
 a. there exists an NR requested by the same user and
 b. all the names exact match the names of the current request and
 c. in pending for review or payment pending status

From @yuisotozaki

Hi Eve, the design team spent a good chunk of time reviewing the scenarios and I have a new text for scenario 1. The designers were OK with scenario 2 proposal.

Scenario 1 Title
Changes Temporarily Unavailable

Scenario 1 Message
Another user may be making updates to this Name Request and this action is unavailable at this time. Please try again later. Typical reasons are as follows:

- Another user may be editing this name request
- BC Registries staff may be currently reviewing this Name Request
- Status of this Name Request has changed

@eve-git
Scenario 1 updated as requested
image

Scenario 2 - updated as requested
image

For Scenario 1, there is an extra line above "For assistance..." that should be removed. Low priority though.


Question: Deprecate use of $smoothLine as well as $plotStyle='lineMarker' in Chart/DataSeries for ScatterCharts?

We can now use mitoteam/jpgraph, which is maintained in Composer, in place of jpgraph/jpgraph. Closing this issue.

Deprecate use of $smoothLine as well as $plotStyle='lineMarker' in Chart/DataSeries for ScatterCharts
This is:

- [x] a bug report
- [x] a feature request
- [ ] **not** a usage question (ask them on URL or URL

N.B. DataSeries->plotStyle property value STYLE_LINEMARKER (='lineMarker') is currently examined only in Chart/Renderer/JpGraph.php, whose use is deprecated. Otherwise, DataSeries property value 'lineMarker' is unused.

Similarly, the DataSeries->smoothLine property is never tested, but its presence potentially confuses users who mistakenly use this (identically named) property and its (identically named) getter/setter methods in both DataSeries as well as DataSeriesValues.

### What is the expected behavior?
When the user assigns the 'smoothLine' property to a DataSeriesValues array [via $DSV->set_smoothLine(true) ] then the Chart line should be displayed as a smooth Bezier line regardless of any property in its DataSeries, (in my opinion).

### What is the current behavior?
If DataSeriesValues->smoothLine == TRUE its line will be straight if its DataSeries->plotStyle == 'lineMarker' (regardless of the value of DataSeries->smoothLine,) This is because Chart/Writer/Xlsx qualifies its writing of the element with a logical AND of '$plotType ===DataSeries::TYPE_SCATTERCHART' and '$plotStyle' === 'smoothMarker'. Therefore, a DataSeriesValues array with a specified property value of TRUE for 'smoothLine' will not be expressed in the xml as unless the DataSeries' plotStyle == 'smoothMarker'. Instead, will be absent from the xml.

### What are the steps to reproduce?
In example file samples/Chart/33_Chart_create_scatter2.php, change as follows:
WAS: DataSeries::STYLE_SMOOTHMARKER // plotStyle

1. IS: DataSeries::STYLE_LINEMARKER // plotStyle
This changes the original smooth (Bezier) line nature of the first DataSeriesValues series to a straight line point connection (which is the default behavior).

2. IS: // DataSeries::STYLE_LINEMARKER // plotStyle
Same result.

Neither result is affected by any value of property DataSeries->smoothLine, including null.

Please provide a Minimal, Complete, and Verifiable example of code that exhibits the issue without relying on an external Excel file or a web server:

php
<?php

require __DIR__ . '/vendor/autoload.php';

// Create new Spreadsheet object
$spreadsheet = new \PhpOffice\PhpSpreadsheet\Spreadsheet();

// add code that show the issue here...

If this is an issue with reading a specific spreadsheet file, then it may be appropriate to provide a sample file that demonstrates the problem; but please keep it as small as possible, and sanitize any confidential information before uploading.

### What features do you think are causing the issue

- [ ] Reader
- [x] Writer
- [ ] Styles
- [ ] Data Validations
- [ ] Formula Calulations
- [x] Charts
- [ ] AutoFilter
- [ ] Form Elements

### Does an issue affect all spreadsheet file formats? If not, which formats are affected?
Xlsx is affected

### Which versions of PhpSpreadsheet and PHP are affected?
1.24 7.3

JpGraph is not deprecated. It is just not maintained in Composer, and the version there is not Php8 compatible (maybe not even some versions of 7) so we can't allow its use in our test suite. However, you can manually substitute the current version of that product for the version that Composer brings in, and it should work just fine. I have no idea what it's doing with lineMarker, but am reluctant to make it stop working (under the uncertain assumption that it works now).

As for smoothLine, that was one of your first requests that I worked on. It seemed to me at the time that Excel required it at both the DS and the DSV levels. I can look into it again.

This annoying behavior tripped me up while I was testing Axis cases.

Thank you for the JpGraph clarification - I had hoped to use it, and I now feel free to try it - if I can find a compatible version.

[N.B. I understand removing properties is a breaking change, but having two identically-named properties in two similarly-named classes (DataSeriesValues & DataSeries) with identical getter/setter methods is seriously confusing. At least to me.]

I repeat my previous request to remove DataSeries property $smoothLine and its getter/setter methods. In a ScatterChart, the accidental assignment of 'lineMarker' to $plotStyle will suppress a $DSV whose $smoothLine property is true, and the user will have a difficult time discovering the reason for the resultant straight-line. To prevent this unfortunate circumstance, I propose that Writer/Xlsx change the qualification for the call to startElement('c:smooth') in the "// Values" section of function writePlotArea() from:

WAS: if ($groupType === DataSeries::TYPE_SCATTERCHART && $plotGroup->getPlotStyle() === 'smoothMarker')

IS: if ($groupType === DataSeries::TYPE_SCATTERCHART)

This will allow a $DSV->smoothLine property value of TRUE to be expressed correctly in the xml regardless of its containing $DS->plotStyle value, (which is the correct behavior in my opinion.)

Under what circumstances would you "accidentally" set the style to lineMarker? It sounds like you're trying to get PhpSpreadsheet to ignore something the programmer has explicitly requested - it certainly isn't used as any kind of a default. Perhaps, after looking at the finished product, smooth lines aren't desired after all, and it's easier to change the DataSeries property to lineMarker in one place than it is to remove the smooth declarations from all the DSVs? It is possible that you can make a stronger case for using the c:smooth tag for "not lineMarker"; I would still need to think about that.

Issue #2783 has more details about how to use JpGraph.

Excel describes 'smooth line' charts as well as 'line charts'. It is not a stretch to associate those choices with 'lineMarker' and 'smoothMarker'. But what if you want a mix?

I submit that a programmer who assembles a ScatterChart may simply ignore assigning any value to the DS->plotStyle property since it is possible to desire an XY chart with both smooth lines and straight lines, and maybe no lines. So the programmer omits it, and its value is null. If the programmer has assigned TRUE to any DSV->smoothLine property, she will be mystified why it doesn't work. (We'll know, but she won't have much of a clue.) Changing Writer masks her (unfortunate) choice of not assigning 'smoothMarker' to DS->plotStyle.

Thank you for the JpGraph link.

Sorry, I don't think you've made a case for any code changes. You have made a case for documentation changes, and 2950, which will close this ticket when merged, will document (in both places) that scatterLines needs to be set at both the DS and DSV level.

Ok. Your call certainly. Documenting will help avoid the possibility shown below using sample/Chart/33_Chart_create_scatter2.php.

Case 1: As written - behavior as expected

$series = new DataSeries(
 DataSeries::TYPE_SCATTERCHART, // plotType
 ...
 DataSeries::STYLE_SMOOTHMARKER // plotStyle
);

image

Case 2: Modified to comment the SMOOTHMARKER designation. I suggest this might be reasonable thinking since, after all, I know one line will be smooth, one straight, and one has no line at all. To ensure the correct display, I have specified each DSV's line charactieristics. Since I don't want to have the series combination ($series) override each DSV assignment, I comment the last line,)
The smooth line is now a straight line.

$series = new DataSeries(
 DataSeries::TYPE_SCATTERCHART, // plotType
 ...
 // DataSeries::STYLE_SMOOTHMARKER // plotStyle
);

image

Chart read write HTML and PDF
This is:

- [x] a bug report
- [ ] a feature request
- [ ] **not** a usage question (ask them on URL or URL

### What is the expected behavior?
The pdf and html file with the charts

### What is the current behavior?

32 Chart read write HTML
15:13:01 Load Test from Xlsx file 36writeLineChart1.xlsx
15:13:01 Iterate worksheets looking at the charts
15:13:01 Worksheet: Data
15:13:01 chart1 - "Sales"
15:13:01
15:13:01 lineChart
Fatal error: Uncaught TypeError: JpGraphException::defaultHandler(): Argument #1 ($exception) must be of type Exception, Error given in C:\xampp.8.0.18\htdocs\phpexcel\vendor\jpgraph\jpgraph\lib\jpgraph\src\jpgraph_errhandler.inc.php:158 Stack trace: #0 [internal function]: JpGraphException::defaultHandler(Object(Error)) #1 {main} thrown in C:\xampp.8.0.18\htdocs\phpexcel\vendor\jpgraph\jpgraph\lib\jpgraph\src\jpgraph_errhandler.inc.php on line 158

32 Chart read write PDF
15:07:46 Load Test from Xlsx file 36writeLineChart1.xlsx
15:07:46 Iterate worksheets looking at the charts
15:07:46 Worksheet: Data
15:07:46 chart1 - "Sales"
15:07:46
15:07:46 lineChart
Fatal error: Uncaught TypeError: JpGraphException::defaultHandler(): Argument #1 ($exception) must be of type Exception, Error given in C:\xampp.8.0.18\htdocs\phpexcel\vendor\jpgraph\jpgraph\lib\jpgraph\src\jpgraph_errhandler.inc.php:158 Stack trace: #0 [internal function]: JpGraphException::defaultHandler(Object(Error)) #1 {main} thrown in C:\xampp.8.0.18\htdocs\phpexcel\vendor\jpgraph\jpgraph\lib\jpgraph\src\jpgraph_errhandler.inc.php on line 158

### What are the steps to reproduce?
Run 32_Chart_read_write_HTML.php
Run 32_Chart_read_write_PDF.php

### What features do you think are causing the issue

- [ ] Reader
- [ ] Writer
- [ ] Styles
- [ ] Data Validations
- [ ] Formula Calulations
- [x] Charts
- [ ] AutoFilter
- [ ] Form Elements

php8.0
"phpoffice/phpspreadsheet": "^1.23",
"mpdf/mpdf": "^8.1",
"dompdf/dompdf": "^1.2",
"tecnickcom/tcpdf": "^6.4",
"jpgraph/jpgraph": "^4.0"


Question: Opened connections are not closed?

Hi all,

We've recently added an **experimental** Host Monitoring Plugin (PR #764) to the driver, called the Host Monitoring Plugin v2. More information can be found here. The Host Monitoring Plugin v2 is functionally equivalent to the Host Monitoring Plugin and they are configured using the same parameters. The Host Monitoring Plugin v2 was created to address the issues that you all have been experiencing, and it is meant to be a more stable version of the Host Monitoring Plugin.

As it is currently an experimental plugin, please note that it should be tested before being used in a production environment. This plugin can be tried out by using the snapshot build `aws-advanced-jdbc-wrapper-2.3.2-20231213.213138-10.jar`. For more information on using a snapshot build, see this page. If you are able to test out the plugin, please let us know if you encounter any errors or if this plugin was able to resolve the issues you've encountered.

Thank you!

Opened connections are not closed
### Describe the bug

Hey 👋 ,
I've started using this driver with a service that was originally using pgJDBC, I just replaced the driver with AWS JDBC driver.

there is a behaviour that I'm seeing which is very consistent where opened connections keep increasing and never closed which causes a OOM.

I'm using Spring JDBC template to issue SQL queries to the DB.

### Expected Behavior

connections are closed and no OOM is thrown

### What plugins are used? What other connection properties were set?

auroraConnectionTracker,efm,failover,iam

### Current Behavior

org.postgresql.jdbc.PgConnection seem to not get closed

I have enabled driver logs and I can see this

Opened Connections Tracked: [
[
 [ instance-1.rds.amazonaws.com:5432:
 {
 org.postgresql.jdbc.PgConnection@245253d8,
 org.postgresql.jdbc.PgConnection@43a9541c
 ....
 }]
]

which initially starts with connections = number of connections configured in the hikari pool (15), but after a couple of hours the number keeps increasing (reaching 100+) till a OOM is thrown.

this service have very little memory pressure yet it throws a out of heap space every couple of hours.

### Reproduction Steps

Using the following

AWS JDBC Version: 2.2.4
Postgresql runtime driver dependency: 42.6.0
Spring boot: 2.7.11
pg-Aurora: 15.3 (2 read instances & 1 writer)
Hikari: 4.0.3

Driver config:
spring:
 datasource:
 type: com.zaxxer.hikari.HikariDataSource
 url: ${DB_URL}
 username: ${DB_USERNAME}
 driver-class-name: software.amazon.jdbc.Driver
 hikari:
 data-source-properties:
 wrapperPlugins: auroraConnectionTracker,efm,failover,iam
 wrapperDialect: aurora-pg
 wrapperLogUnclosedConnections: true
 wrapperLoggerLevel: ALL
 exception-override-class-name: software.amazon.jdbc.util.HikariCPSQLException
 max-lifetime: 300000
 connection-timeout: 1200
 validation-timeout: 1000
 maximum-pool-size: 15

and using the sslmode=verify-full & AWS RDS global-bundle.pem

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### The AWS Advanced JDBC Driver version used

2.2.4

### JDK version used

oracle-jdk:17 (17.0.6+9-LTS-190)

### Operating System and version

Red Hat Enterprise Linux 8.7 (Ootpa)

Hi @abdoofathy95,

Thanks for reaching out and raising this issue.

We'll take a look at this and keep you updated as we investigate.

In the meantime, could you provide the following additional information if possible?
- Driver logs, which can be enabled as seen here.
- A bit of sample code, to get a better idea of how you're using the driver.

Thank you for your patience!

Hi @crystall-bitquill,
* for the driver logs, as mentioned above, the Opened Connections Tracked log shows a list that keeps growing (never shrinks), or are you interested in another bit of the log ?
* a sample code:

spring:
 datasource:
 type: com.zaxxer.hikari.HikariDataSource
 url: ${DB_URL}
 username: ${DB_USERNAME}
 driver-class-name: software.amazon.jdbc.Driver
 hikari:
 data-source-properties:
 wrapperPlugins: auroraConnectionTracker,efm,failover,iam
 wrapperDialect: aurora-pg
 wrapperLogUnclosedConnections: true
 wrapperLoggerLevel: ALL
 exception-override-class-name: software.amazon.jdbc.util.HikariCPSQLException
 max-lifetime: 300000
 connection-timeout: 1200
 validation-timeout: 1000
 maximum-pool-size: 15

class Repository {
...
private final JdbcTemplate jdbcTemplate;
 public int[] query(paramArray) {
 return jdbcTemplate.batchUpdate("SQL", paramArray);
 }
}
....

Hey, this seems related mainly to using efm plugin, when disabled and instead using the following plugins
`auroraConnectionTracker,failover,iam` not OOMs are observed. I didn't get the time to properly look at the exact root cause tho.

Hi @abdoofathy95,

To update you, it seems like the additional connections you're seeing in the connection tracker comes from the Hikari pool. When idle connections reach the maxLifetime, they are closed and new connections are made. The connection tracker is logging these closed connections. However, all of these closed connections are weak references and can be removed by garbage collection. Have you made any changes to any garbage collection settings? If not, it is unlikely that the OOM error was caused by the connections visible in the tracker, as they should be garbage collected before the OOM error occurs. This occurs regardless of which plugins are used.

You mentioned you didn't get the OOM error when efm was disabled. Would you be able to try the following plugin combinations to confirm where the error is?
1. No plugins - this can be done by setting the `wrapperPlugins` property to an empty string (the driver has several plugins enabled by default).
2. EFM only

In addition, I have a few questions:
1. Are you able to see how much free memory you have before and after starting your application?
2. Are you ablt to tell at what rate the amount of free memory you have available is decreasing?
3. Are any failover events occurring?

Hey @crystall-bitquill,
thank you for your reply,
I'm using the following flags
`-XX:+UseG1GC -Xms1536m -Xmx1536m -XX:MaxMetaspaceSize=256M -Xss2m`.

I will try to disable all the plugins and see the behaviour.

1. not sure I get what you mean here by before starting the application, it's a pod that as soon as it gets deployed, the application starts.
2. this is the JVM heap metric URL
3. There are no failover events occurring.

Hi @abdoofathy95,

To update you, similar issues (#675) have been reported regarding a memory leak in the Host Monitoring plugin.

We've recently added an **experimental** Host Monitoring Plugin (PR #764) to the driver, called the Host Monitoring Plugin v2. More information can be found here. The Host Monitoring Plugin v2 is functionally equivalent to the Host Monitoring Plugin and they are configured using the same parameters. The Host Monitoring Plugin v2 was created to address the issues that you and others have been experiencing, and it is meant to be a more stable version of the Host Monitoring Plugin.

As it is currently an experimental plugin, please note that it should be tested before being used in a production environment. This plugin can be tried out by using the snapshot build `aws-advanced-jdbc-wrapper-2.3.2-20231213.213138-10.jar`. For more information on using a snapshot build, see this page. If you are able to test out the plugin, please let us know if you encounter any errors or if this plugin was able to resolve the issues you've encountered.

Thank you!

Hi @abdoofathy95 ,

Heads up that the aforementioned experimental Host Monitoring Plugin is now available on our latest release of the driver.

Also wanted to reiterate that it is currently an experimental plugin that should be tested before being used in a production environment.

If you are able to test out the plugin, please let us know if it resolves these issues or if there are any other issues encountered.

I'll be closing this ticket. But if this issue still persists or if you encounter any other issues, please feel free to create a new ticket or re-open this one.

Thank you

Memory Leak in EFM Plugin
### Describe the bug

In our application we've two aurora databases each with an own jdbc connection. Through our monitoring we found that our pods randomly restarted due to out of memory. After some investigation we found out that the `software.amazon.jdbc.plugin.efm.MonitorImpl` of one of our jdbc connections stopped working and we don't saw any exceptions. In this case it collects `software.amazon.jdbc.plugin.efm.MonitorConnectionContext` objects in the `newContexts` queue. In one of our heap dumps we found out that in the `newContexts` queue were over 4000000 objects.

This screenshot from one of our heap show that the first instance is `stopped=true` with over 411MB of objects in the `newContexts` queue. The second one is the one which is working it no objects in the `newContexts` queue.
<img width="1263" alt="grafik" src=" URL

### Expected Behavior

No memory leak.

### What plugins are used? What other connection properties were set?

auroraConnectionTracker,failover,efm,iam

### Current Behavior

Randomly the Monitor stopps and objects are stored in `newContexts` but they won't ever be removed.

### Reproduction Steps

We couldn't find any reproducer as we just observed this behavior randomly without any pattern.

### Possible Solution

At least catch and log `RuntimeException` and return an appropriate `ConnectionStatus` object in `software.amazon.jdbc.plugin.efm.MonitorImpl#checkConnectionStatus` to avoid that uncaught runtime exceptions stop the monitoring thread without shutting down the monitor completly.

### Additional Information/Context

_No response_

### The AWS Advanced JDBC Driver version used

2.2.2

### JDK version used

zulu 11.0.17

### Operating System and version

Distroless Docker Image at AWS EKS

Hi @chr-fritz,

Thanks for reaching out and raising this issue.

We'll take a look at this and keep you updated as we investigate.

Thank you for your patience!

Hi @chr-fritz,

We've recently merged some changes (PR #676) that updated our monitoring implementation and added additional logging. I understand that you aren't able to reproduce this consistently, but would you be able to check out our snapshot build with driver logs enabled (logs can be enabled as seen here) and let us know how that goes?

Thank you!

Hey @crystall-bitquill,

I'm also experiencing the same issue reported here. I've using a snapshot build and getting the following log line (identifying details removed). I'm thinking we need to include the exception type in the log line.

WARNING: Continuing monitoring after unhandled exception was thrown in monitoring thread for node -02. .us-west-2.rds.amazonaws.com: null
Nov 14, 2023 9:23:23 PM software.amazon.jdbc.plugin.efm.MonitorImpl run

Hi @andrecloutier-pd,

Thanks for trying out the snapshot build and for letting us know about the logs. There were some changes merged recently (PR #718) that make the logger message include the entire exception. Could you kindly check out the latest snapshot build and let us know how that goes?

Thank you!

costly

I wonder if this is the same issue? Two days ago we got a budget warning from AWS. Something had generated a few gigabytes (or hundreds of dollars depending on how you want to measure it) of cloudwatch log entries. The CPU was spinning on 100% generating the output in the image as fast as it could.

No wonder AWS is big business for Amazon

Hi @kimble, thank you for reaching out with this information.

The driver logs thrown by the monitoring thread are necessary to inform the user of potential monitoring problems. This info can help the user and team diagnose and address issues that may arise. However, we understand that the volume of logs generated by this particular scenario was greater than you expected. To address this, we have created PR #751, which lowers the logging level for the particular messages you are seeing to avoid potential surprises for other customers. Thank you for raising your concern. Please note that you can also filter the logs using the properties file. For example, to filter out the logs in the posted image you can add the following line to your `.properties` file:

software.amazon.jdbc.plugin.efm.MonitorImpl.level=OFF

Regarding the memory leaks issue, we have made improvements to the monitoring objects to ensure they are synchronized and the driver does not cache references to them in PR #718.
We will keep you updated for when we create an official release containing this fix.

In the meantime, could you please help us further improve our driver by checking out our snapshot build containing changes from PR #751 and PR #718, and send us the unhandled exceptions your application is throwing so we can triage it?

Reducing the log level will probably just hide the problem that there is a thread consuming 100% printing the lines from my screenshot over and over as fast as it can.

Not sure what you mean by this?

Hi @kimble,

1. The logs will only be printed if you choose to enable `FINEST/DEBUG` level logs. You can also disable logging for that particular thread.
2. We have released v2.3.1 containing fixes from #751 and #718. Could you please give it a try? If the issue persists can you please send us the driver logs related to the issue?

Hi all,

Heads up that the aforementioned experimental Host Monitoring Plugin is now available on our latest release of the driver.

Also wanted to reiterate that it is currently an experimental plugin that should be tested before being used in a production environment.

If you are able to test out the plugin, please let us know if it resolves these issues or if there are any other issues encountered.

Thank you


Question: update_bindings failed?

Should be fixed by the recently merged #1100 - let me know if that isn't the case.

update_bindings failed
I met the issue on ubuntu 20.04 and python3.8
And met the issue:
aceback (most recent call last):
 File "/root/ompl/py-bindings/generate_bindings.py", line 967, in
 globals()'ompl_'+module+'_generator_t'
 File "/root/ompl/py-bindings/generate_bindings.py", line 626, in __init__
 replacement)
 File "/root/ompl/py-bindings/ompl/bindings_generator.py", line 142, in __init__
 self.std_ns = self.mb.namespace('std')
 File "/usr/local/lib/python3.8/dist-packages/pyplusplus/module_builder/module_builder.py", line 267, in namespace
 , recursive=recursive )
 File "/usr/local/lib/python3.8/dist-packages/pygccxml-2.2.1-py3.8.egg/pygccxml/declarations/namespace.py", line 121, in namespace
 recursive=recursive)
 File "/usr/local/lib/python3.8/dist-packages/pygccxml-2.2.1-py3.8.egg/pygccxml/declarations/scopedef.py", line 464, in _find_single
 found = matcher.get_single(decl_matcher, decls, False)
 File "/usr/local/lib/python3.8/dist-packages/pygccxml-2.2.1-py3.8.egg/pygccxml/declarations/scopedef.py", line 92, in get_single
 raise runtime_errors.multiple_declarations_found_t(decl_matcher)
pygccxml.declarations.runtime_errors.multiple_declarations_found_t: Multiple declarations have been found. Matcher: [(decl type==namespace_t) and (name==std)]

Is there anyone that can help me to solve it ?
Words fail me to express the heartfelt gratitude

Some updated:
pip3 list
Package Version
---------- -------
numpy 1.17.4
pip 20.0.2
pygccxml 2.2.1
pyplusplus 1.8.5

 $castxml --version
castxml version 0.5.0

CastXML project maintained and supported by Kitware (kitware.com).

clang version 13.0.0
Target: aarch64-unknown-linux-gnu
Thread model: posix

This is the same issue as reported in #981, please keep discussion there.

Cannot build Python bindings on NVidia Jetson Orin
Hi all, recently I tried to build OMPL Python bindings on Nvidia Jetson Orin but faild.
I have tried OMPL 1.5.2, OMPL 1.6.0 and the main(d9c9566) branch which updated recently, but still get the same error while compiling:
image
I have tried the provied script **install-ompl-ubuntu.sh** and the **Dockerfile**, respectly under Ubuntu 20.04 and 22.04 in docker, and also tried different version of **Castxml** (the latest **0.5.0** and **0.3.6**).

I notice the similar issues and tried, but still met that error.

thx for reading and hope your reply

How much RAM does your platform have? Building the Python bindings can be rather RAM hungry - at least 6GB is recommended.

Hi Zachary @zkingston , appreciate your timely reply. Our target platform is Nvidia Jetson AGX Orin Developer Kit, the memory is 32 GB 256-bit LPDDR5.

Thx for your reply.
As for the RAM, I think that's quite enough as shown below,
image

oops, I also met the exactly same issue ! It f**ked me for nearly a month.
Quite strange, that Cannot build on ubuntu Arm platform.
@zkingston , could you help to check it ?
Thanks

Can you please post the full output of trying to compile the docker image on your platform? #958 also had issues compiling on ARM.

Here is the output of building the docker container,

log.out.txt

The error appears to be with CastXML on ARM, a number of these errors appear, indicating that it failed to generate output, and thus the rest of the Python binding pipeline failed:

In file included from /tmp/tmpapu0i3zy.h:1:
In file included from ./bindings/base.h:1145:
In file included from /usr/include/eigen3/Eigen/Core:214:
/usr/include/eigen3/Eigen/src/Core/arch/NEON/PacketMath.h:3096:10: error: use of undeclared identifier 'vrndn_f32'
{ return vrndn_f32(a); }

I'm surprised this error occurred with the latest CastXML - it appears to have been fixed in the past: URL
Can you verify that if you install the latest CastXML in Docker if things work?

@zkingston I've just verified castxml 0.5.0 from the official URL (the "castxml_v0.5.0-linux-aarch64.tar.gz"),

and updated the Dockerfile (only comment out the castxml and use COPY command),
ompl.Dockerfile.txt

After all, the output shows like,
log0206.out.txt

I believe this is now an issue with pygccxml ( URL - please make sure the most recent version is installed in the docker as well. Also take a look at the discussion in URL - the same error message appears there.

HI, just followed the issue these days and tried your tips , but all failed! All failed.
We had absolutely 100% sure that we use the latset pygccxml(it's 2.2.1) and the latest commit of the pygccxml is May.25.2021
And
#725 is the same issue , but no one can solve it!!!!

@zkingston I am sure that the latest pygccxml is installed in the docker.
Also, I tried the solutions mentioned in #725 , but unfortunately they doesn't work.

I do not have an ARM platform to test on to replicate this, could you insert `traceback.print_exc()` at the bottom of `generate_bindings.py` to see where exactly the error is occuring?