Question: [Bug]: Electron helper high cpu usage?

@brandon-doornbos, thanks. That RC build is based on recently released @electron 16.0.6 which includes @chromium bump so I thought maybe the issue got fixed somewhere upstream in @chromium. So will be releasing a new version built on old/v15 @electron.

Doesn't happen to me on Linux for some reason.

[Bug]: Electron helper high cpu usage
### Preflight Checklist

- [X] I have read the Contributing Guidelines for this project.
- [X] I agree to follow the Code of Conduct that this project adheres to.
- [X] I have searched the issue tracker for a feature request that matches the one I want to file, without success.

### Electron Version

16.0.1

### What operating system are you using?

macOS

### Operating System Version

macOS big sur 11.5.2

### What arch are you using?

x64

### Last Known Working Electron version

_No response_

### Expected Behavior

Observing system load should be fine I think.

 URL

the module works fine itself, but when i run it with electron or as a fork process it impact cpu usage

### Actual Behavior

image

the electron helper is about 99% usage

### Testcase Gist URL

 URL

### Additional Information

image

bash
git clone URL
cd revealer
npm i
npm run dev

Also starting from @electron v16 facing an abnormally high CPU load issue, see URL So had to downgrade to v15. Could not yet narrow down the scope.

Thanks for reporting this and helping to make Electron better!

Because of time constraints, triaging code with third-party dependencies is usually not feasible for a small team like Electron's.

Would it be possible for you to make a standalone testcase with only the code necessary to reproduce the issue? For example, Electron Fiddle is a great tool for making small test cases and makes it easy to publish your test case to a gist that Electron maintainers can use.

Stand-alone test cases make fixing issues go more smoothly: it ensure everyone's looking at the same issue, it removes all unnecessary variables from the equation, and it can also provide the basis for automated regression tests.

I'm adding the `blocked/need-repro` label for this reason. After you make a test case, please link to it in a followup comment.

Thanks in advance! Your help is appreciated.

@deepak1556

Do you think this thread URL is related to the current bug?

我目前用的是 Linux arch芯片的电脑，使用electron做的应用出现了这个问题，有什么解决办法吗？
#######
I am currently using a computer with Linux arch chip. This problem occurs in the application made with electron. Is there any solution?

High CPU usage as of v4.13.0
Hi, I upgraded to `v4.13.0` recently and noticed that my fans were spinning even though I wasn't doing any CPU intensive tasks. Upon inspections it looks like one of the threads belonging to ElectronMail were the cause:
image

Maybe it's due to some kind of continuous event loop?

OS: Arch Linux

It's a renderer/web/@chromium process but hard to say just by the screenshot which part of the functionality it's related to (@protonmail web clients or app's UI). I'm not seeing any intensive CPU load at my side (Linux).

I think the issue started after I enabled the "custom actions" menu, but that is blind speculation on my part as it could've been a coincidence.

I can help by debugging the code on my computer. I cloned the code and installed the dependencies, but I can't find a way to run the app without building it so that I can use debugger tools to profile the code.

$ npm run start:electron:dev

Error launching app
Unable to find Electron app at ./app-dev/electron-main.js

Can you guide me through the debug process? I have experience with JavaScript and NodeJS, but I am not really familiar with other tools, especially TypeScript.

I thought maybe I missed properly disconnection the `MutationObserver` thing which might potentially cause the issue. But I just debugged it and can confirm it gets well disconnected (here is the code line). The minor optimization I just pushed is 9af8f4c665d6eae57a4b0a2df7ef7ae73dd454b0.

It's the same build steps as listed in readme (excluding the packaging step), but at the end run `yarn assets:dev && yarn build:dev`. Then you start it by `yarn start:electron:dev`. There is at the moment no hot reload thing for the web stuff (dropped it long time ago and didn't get back yet).

Build command: `yarn && yarn assets:dev && yarn build:dev && yarn clean:prebuilds && npm exec --package=electron-builder -- electron-builder install-app-deps --arch=x64`

Thanks for the instructions, but for some reason yarn is giving me errors, I ran the command from your post as well as the one from README: URL (output is too long so I pasted on an external site)

I missed the fact that you will need to build @protonmail clients by running `yarn assets:dev`. Better download those clients by URL (taken from URL ie v4.13.0 release build jobs). Then unpack the archive to some place, rename the `app => app-dev` folder, and then copy to the project folder.

I did what you asked, and ran `yarn build:dev`, but it's still not working...

TheDcoder@arch /m/d/P/J/ElectronMail (master)> yarn start:electron:dev
yarn run v1.22.17
$ electron ./app-dev/electron-main.js
Error launching app
Unable to find Electron app at /mnt/data_partition/Projects/JavaScript/ElectronMail/app-dev/electron-main.js

Cannot find module '/mnt/data_partition/Projects/JavaScript/ElectronMail/app-dev/electron-main.js'
Require stack:
- /mnt/data_partition/Projects/JavaScript/ElectronMail/node_modules/electron/dist/resources/default_app.asar/main.js
-
error Command failed with exit code 1.
info Visit URL for documentation about this command.

TheDcoder@arch /m/d/P/J/ElectronMail (master)> ls app-dev
assets electron-main electron-preload web

I think we can debug this more efficiently if we can communicate in real-time, do you use IRC or Matrix?

My bad, I didn't update the `start:electron / start:electron:dev` npm scripts after recent `./app/electron-main.js => ./app/electron-main/index.cjs` rename. So use direct run, like `npx electron ./app-dev/electron-main/index.cjs`. Done now in 92a8834b7f5491a5bc639e96acb873eb5a09e086.

Direct run worked... but unfortunately the native build artifacts were compiled with an older version of node, so I can't use them:

Error: The module '/mnt/data_partition/Projects/JavaScript/ElectronMail/node_modules/spellchecker/build/Release/spellchecker.node'
was compiled against a different Node.js version using
NODE_MODULE_VERSION 93. This version of Node.js requires
NODE_MODULE_VERSION 99. Please try re-compiling or re-installing
the module (for instance, using `npm rebuild` or `npm install`).
 at process.func [as dlopen] (node:electron/js2c/asar_bundle:5:1800)
 at Object.Module._extensions..node (node:internal/modules/cjs/loader:1199:18)
 at Object.func [as .node] (node:electron/js2c/asar_bundle:5:1800)
 at Module.load (node:internal/modules/cjs/loader:988:32)
 at Module._load (node:internal/modules/cjs/loader:829:12)
 at Function.c._load (node:electron/js2c/asar_bundle:5:13331)
 at Module.require (node:internal/modules/cjs/loader:1012:19)
 at require (node:internal/modules/cjs/helpers:94:18)
 at Object. (/mnt/data_partition/Projects/JavaScript/ElectronMail/node_modules/spellchecker/lib/spellchecker.js:3:16)
 at Module._compile (node:internal/modules/cjs/loader:1116:14) {
 code: 'ERR_DLOPEN_FAILED'
}

I will try building ProtonMail myself.

A native dependencies. You need to run `yarn clean:prebuilds && npm exec --package=electron-builder -- electron-builder install-app-deps --arch=x64` (listed above).

Oops, my bad, I missed that command. I ran it again and the app is running :+1:

I have encountered another issue though, I am getting this error while logging in;

Failed to resolve the "web-client" bundle location by " URL API entry point value

My fan is not spinning at full speed yet, so the resource-consuming loop isn't triggered at this point.

Make sure you properly unpacked the @protonmail web clients (see URL After doing that unpacking/renaming you need to run `yarn assets:dev` again so the script takes client from the ./output/git/backup and puts it into ./app-dev/webclient.

At this point it's only the app stuff is loaded, no @protonmail involving yet.

Awesome, thanks for assistance, I am able to run the app now.

But I couldn't reproduce the 100% CPU usage... so I tested with the pre-built release version and it still is eating up 100% of the CPU. It only happens once I load an email, so I guess it has to do something with the specific version/build of the ProtonMail stack?

@vladimiry Did you test the release build? Perhaps you might be able to reproduce it too.

Sorry for not explaining properly, I mean the exact binaries, I think a very specific version of ProtonMail stack might be responsible...

I too have high CPU usage since the update.
I'm on Windows 10 with a i7700k and ElectonMail is is using 13-17% CPU, even when its closed to the tray.

Can you test the build shared in URL (it's v4.13.0 but with downgraded @electron).

@vladimiry This build is working without performance issues! Hardly any CPU is being used during idle now :)

Great, so you have got a workaround for now. It means that the issue lies somewhere in @electron since 16.0.0-beta.8 => 15.3.1 downgrade did the trick.

Just confirming that the build is working for me too on Windows 10.

So it's not system-specific as the issue reported to be happening in Linux and Windows 10.

I will consider publishing a new release with downgraded @electron if more reports get posted here. But would prefer to first make a new build based on a stable @electron v16 (it's going to be released next week).

I am experiencing high cpu usage on Windows 11 as well since last update

Ok, I wait then for @electron v16 stable release which gets published this week. Then I ask to test the new build and then I release the v15 or v16 @electron-based version depending on the testing outcome.

I'm asking everyone affected by abnormal CPU load of v4.13.0 to test the build shared in URL

WIP version 4.13.1 from mega link still causing 10% (high) cpu usage all the time on Windows 10.

Thanks. So a stable/today's @electron v16 reelase is also affected. It's at the moment unclear what exactly causes the abnormally high CPU usage between different @electron versions and I was not able to locate related open issues in URL but since a renderer process is affected the cause is likely somewhere in @chromium integrant. So it looks like we had to stick to @electron v15 for now.

Hopefully, a final testing build is uploaded here. Going to publish it as v4.13.1 release when someone confirms that it doesn't cause an abnormally high CPU usage.

v4.13.1-wip-2 no more abnormal cpu usage for me on Windows 10.

@vladimiry I tested the 2nd wip and I see constant 2-3% usage by two of the processes, maybe this is an unrelated issue but it is there. It was not present in `v4.13.0` with downgraded electron, it had 0% usage during idle.

@TheDcoder the recent build I shared (v4.13.1-wip-2) is built on @electron 15.3.2. The previously shared was 15.3.1-based and verified as not causing the issue. I'm ok to wait for one/two days before publishing the recently shared build as a release so we could get additional feedback from you or someone else.

For me v4.13.1-wip-2 (built on @electron 15.3.2) works fine - i see no idle cpu usage

Can confirm that v4.13.1-wip-2 also fixes the high CPU usage for me (Arch Linux).

The v4.13.1 RC build assembled from the `master` branch is uploaded here. Hope someone confirms that there is no issue. It should be fine as it's based on the same @electron v15.3.2 but comes with updated @protonmail web clients.

Any more feedbacks? I'm about to publish the recently shared build as a new release.

Can someone who had CPU usage problem before to test this v4.13.2 release candidate build?

Changelog draft:
- Bundle proton-vpn-settings web client into the app (#470).
- Update @ProtonMail web clients stack. The `session.bin` file produced or modified by the v4.13.2+ app version won't be compatible with the previous app versions.
- Upgrade @electron.

Just tried the RC, and the issue has returned on my Arch Linux machine. One of the processes is again maxing out one thread. (Even after a restart)

It's Manjaro (Arch based). I would not ask someone to test the build if I could reproduce the issue at my side.

To face the issue, do I need to perform a specific action in the app or just start it and login into the proton account?

Please list them here. I don't believe though that @electron contributors will start actively looking into it until the minimal repro project gets provided (which normally hard to do for such kind of issues).

Btw. Notice the below note. So if you use persistent session feature you will have to keep using that RC build or login again if you prefer downgrading to a stable release (new release is coming soon). The reason is that @protomail changed the session credentials storage format in a web clients.

I tried it fresh on my laptop today, and the only thing I did was install the RC pacman file and then login. This did not do anything so then I closed the app and opened it again and upon logging in, one full thread utilized. On my PC, where I already had the previous version, I just installed the RC and it immediately did it upon logging in.

 URL which lists URL which I can't read. The other one I found was for a different version, sorry.

I just can't fathom what would cause this, as it is consistently happening to both my Arch machines but not your Arch based machine.
Is there something else that I could do to help with finding the cause? Are there debugging tools or something?

The app's code need to be compiled in "dev" mode in order to be able to open dev tools (I think it's a web/renderer process i a problematic one). There is no documentation on this matter. It's not a complex process but time-consuming, similar to what described in URL Debugging of the main process can be achieved as listed in URL but I tend to think that it's not the main process causes the problem.

Yes it's clear that it's a renderer process since we got screenshot provided in URL

What about this build? if it runs well I will be publishing it as a new release then.

Yep, no problems.

BTW, i built the thing and a dev build, so I'll see if I can find something.

Hello again, well I could not find anything with just the normal debugging tools, and if I would want to debug electron itself, well, debug symbols. The whole electron repository doesn't even fit on my drive so that's a no.

Let's hope that this gets fixed accidentally someday.

What about this build (built on recently released @electron v18.0.0-alpha.1 from wip-electron-18 branch)? Still causing issue?

Thanks for the test. So it's confirmed that newest @electron v18 alpha is unfortunately also affected.

@brandon-doornbos can you test this build? I has been assembled on recently published @electron v19.0.0-alpha.1. Anyone else is also welcome to test the build.

The high usage does not seem to affect me with this build, so it may actually be fixed. We probably need other people from this thread to confirm it though. (I'm still on the same platform)

Thanks. So hopefully the next app release will be utilizing @electron v19 runtime. If possible please keep using this build (same as v4.13.6 but with bumped @electron).

Good call, after using the build for a while longer, the problem seems to have returned. A renderer process is again fully stressing out a single core :(.

Asking to share the CPU and GPU type used (including integrated graphics) as it might potentially help to narrow down the scope.

Also maybe someone who can reproduce the issue could try running the app with disable-like GPU/rendering-related command-line switches from the URL list (you would run the in-page search by the `gpu / renrerer` keywords to locate those switches).

I've got an R5 1600 and an RX480, so no integrated graphics.

It consistently hits one core when I start the app, then if I switch to another account and close the app, it often fixes itself. So maybe it is related to residual processes.

The behavior is the same in X11 and Wayland.

List of command line arguments I've tried:
- `--disable-mojo-renderer`
- `--disable-gpu`
- `--disable-new-content-rendering-timeout`
- `--disable-partial-raster`
- `--disable-renderer-accessibility`
- `--disable-renderer-backgrounding`
- `--enable-nacl`
- `--enable-oop-rasterization`
- `--extension-process`
- `--font-cache-shared-handle`
- `--font-render-hinting`
- `--instant-process`
- `--renderer-cmd-prefix="valgrind --tool=callgrind"`
- `--disable-accelerated-2d-canvas`
- `--disable-frame-rate-limit`
- `--disable-gpu-compositing`
- `--disable-gpu-driver-bug-workarounds`
- `--disable-gpu-early-init`
- `--disable-gpu-memory-buffer-compositor-resources`
- `--disable-gpu-memory-buffer-video-frames`
- `--disable-gpu-process-crash-limit`
- `--disable-gpu-program-cache`
- `--disable-gpu-rasterization`
- `--disable-gpu-sandbox`
- `--disable-gpu-shader-disk-cache`
- `--disable-gpu-vsync`
- `--disable-gpu-watchdog`
- `--disable-threaded-compositing`
- `--disable-zero-copy`
- `--enable-gpu-debugging`
- `--enable-native-gpu-memory-buffers`
- `--gpu-launcher="valgrind --tool=callgrind"`
- `--ignore-gpu-blocklist`
- `--in-process-gpu`
- `--use-fake-mjpeg-decode-accelerator`

Can someone test the electron v19 based build (build log / changes)? There is a hope that the following changes will make some effect: updated proton clients, updated electron, dropped MutationObserver use in the project.

Well this may finally be it :).
It has not yet shown any signs of excessive resource usage, even after a restart and some usage.

Thanks for the feedback. If possible, please keep using this build for a while. Hopefully, the next app release switches to the recent @electron version.

Still haven't found any issues, after multiple days, many restarts and actual usage.
We probably need a second person from this thread to confirm, but as far as I can see it's fixed.

Also, thank you for continually thinking of us :).

Going to publish this build later today as a release. Can someone do a final test?


Question: Remove/replace references to EC:1.6.1.2?

oh, interesting. This is the solr query:

 URL

Notice that `+ ` is translated ` ` (double space). This is because the `+` is a URL encoding of a space. The solution is to use the right URL escaping when constructing the SOLR urls. We should also check other characters that typically get encoded, e.g. `:`.

Remove/replace references to EC:1.6.1.2
I refreshed the list of obsolete ECs (in #27812), and a new one has been added:

- ERROR: < URL has a definition xref to an obsolete EC: "EC:1.6.1.2"
- ERROR: < URL has a synonym xref to an obsolete EC: "EC:1.6.1.2"
- ERROR: < URL has an xref to an obsolete EC: "EC:1.6.1.2"
- ERROR: < URL has a definition xref to an obsolete EC: "EC:1.6.1.2"
- ERROR: < URL has a synonym xref to an obsolete EC: "EC:1.6.1.2"

This is a bit complicated issue.
EC:1.6.1.1 NAD(P)(+) transhydrogenase (Si-specific) has other name NAD(P)(+) transhydrogenase (B-specific) RHEA:11692
EC:1.6.1.2 NAD(P)+ transhydrogenase (Re/Si-specific) is transferred to
EC 7.1.1.1 proton-translocating NAD(P)(+) transhydrogenase RHEA:47992
EC:1.6.1.3 NAD(P)(+) transhydrogenase RHEA:11692

GO:0008746 NAD(P)+ transhydrogenase activity XREF to EC:1.6.1.1/EC:1.6.1.2/RHEA:11692

... GO:0008750 NAD(P)+ transhydrogenase (AB-specific) activity XREF to EC:1.6.1.2. Also XREF MetaCyc:PYRNUTRANSHYDROGEN-RXN suggests it is about EC:1.6.1.1 and EC:1.6.1.3
... GO:0003957 NAD(P)+ transhydrogenase (B-specific) activity XREF to EC:1.6.1.1 (NOT RHEA:11692)

**For GO:0008746 NAD(P)+ transhydrogenase activity** XREF to EC:1.6.1.1/EC:1.6.1.2/RHEA:11692
- [ ] Obsolete ; this is the general case that corresponds to URL which states "This entry stands for enzymes whose stereo-specificity with respect to NADPH is not known"; this is associated with the same RHEA as 1.6.1.1 and has no genes associated with it. It seems like the stereospecificity is always known.

**Child GO:0008750 NAD(P)+ transhydrogenase (AB-specific) activity** :
- [ ] Change general cross references to EC 7.1.1.1 proton-translocating NAD(P)(+) transhydrogenase and RHEA:47992
- [ ] Change definition to match RHEA
- [ ] Change definition cross reference to RHEA:47992
- [ ] remove the link to "pubs.acs...." in the def xref that (dead link)

**Child GO:0003957 NAD(P)+ transhydrogenase (B-specific) activity** XREF to EC:1.6.1.1
- [ ] Xrefs should be to EC:1.6.1.1 and RHEA:11692
- [ ] Definition should be adjusted to match RHEA's.

Does that seem right ?

Thanks, Pascale

(based on UniProt catalytic activity)
GO:0008746 NAD(P)+ transhydrogenase activity, there are 6 experimental annotations. I reported an AmiGO bug URL Of those 6, there is a mixture of Rhea:47992, Rhea:11692, and Rhea:10824 (incorrect GO annotation?).
GO:0003957 NAD(P)+ transhydrogenase (B-specific) activity The only annotation PntAA is actually of Rhea:47992.

Since GO doesn't have EC:1.6.1.3, this issue may not be relevant. This comment "This entry stands for enzymes whose stereo-specificity with respect to NADPH is not known", I interpret it to mean that EC:1.6.1.3 is different from EC:1.6.1.1 rather than EC:1.6.1.1 is a subclass.

Hi @raymond91125
Is my suggestion above not working What do you propose to do?

Thanks, Pascale

@raymond91125
See @amandamackie 's input in the annotation ticket: URL

GO:0003957 NAD(P)+ transhydrogenase (B-specific) activity 's definition and the parent ''proton transmembrane transporter activity'' should be removed.

Thanks, Pascale

Annotation filter GO class (direct) not working
To reproduce:
 URL
click on GO class (direct) shows Total annotations: 103
(89) NAD(P)+ transhydrogenase activity
(9) NAD(P)+ transhydrogenase (AB-specific) activity
(5) NAD(P)+ transhydrogenase (B-specific) activity
click on any line shows Total annotations: 0

I talked a little with @pkalita-lbl about proximate places to look for the issue. While adding a fix (encoding/decoding), there may also be consequences in the UI filters and bookmarking, in addition to the "main" issue of the query to the Solr server.

There are some oddities that have slipped into the code over the years in response to years of LBL testing (e.g. URL the bug may be a consequence of that (or just a regular "oops" on my part). Either way, a fix should be XSS-safe.

The problem code is probably in the `bbop-manager-golr` package, at URL . A possible place to start might be URL which uses `encodeURI()`. One way to approach this would be by the tests in that package, although that may practically be a slow way forward.

The JS code is old, baroque, and quirky (ooo--`prototype`!). If any problems or confusions come up, it's probably faster to drag me in than directly wrestle with whatever was in my head at the time.

Another good URI-encoding test case

* Go to URL
* Expand the "GO class (direct)" facet and click more
* In the modal search for "ascr"
* Click the "+" button next to "ascr#2 binding"


Question: Kubeflow User Research & Survey Strategy?

Thank you for opening this, the way you've laid out the work to be done and linked other issues is quite helpful!

Kubeflow User Research & Survey Strategy
So far, the Kubeflow community has published a yearly user survey composed of high level questions to gauge how users interact with Kubeflow, what are the most used components, what are the biggest gaps, and how Kubeflow is deployed in production. The survey provides a good snapshot of users demographics and, with the help of freeform questions, it provides a pretty detailed report of specific areas users would like to see improved.

This information has proven very useful to understand what are the most used components; missing capabilities; usability concerns; distributions adoption; etc.

There are two aspects to consider now that the community is maturing and building ever more complex capabilities:

- **Interviews**: the current process still lacks a second critical phase in user research: the interview. User interviews help create a detailed picture of the user personas, understand their needs, validate the survey findings, and drill down into specific usability concerns.
- **Component-specific research**: Working Groups need detailed feedback on existing/new components and features. A (yearly) high-level generic user survey doesn’t help working groups getting specific feedback on what they are building and what users would like to see in the future.

Kubeflow is a very complicated platform, the AI/ML landscape is changing rapidly. We need to adopt a product-led mindset and develop a platform strategy to holistically look at Kubeflow as whole, understand what it is and where it is going, based on consumer’s feedback and understanding. We cannot base our decisions on assumptions, we need a user research approach to validate our intuition and maximize our community effort towards building something that pays off.

I don't want to start in this issue a never ending debate on how we should adopt product thinking (that discussion can happen in a separate issue) but rather have a tactical conversation on how we can improve our current user research initiatves to drive better and more informative outcomes.

In this document URL I list some bullet points on surveys and interview and make some proposal on what we can do in the next months.

cc @kubeflow/wg-notebooks-leads @kubeflow/wg-manifests-leads @kubeflow/wg-training-leads @kubeflow/wg-pipeline-leads @kubeflow/wg-automl-leads @kubeflow/release-team @jbottum @andreyvelich @johnugeorge @james-jwu @akgraner

I also opened a CNCF Service Desk ticket URL to better understand how other CNCF project are doing this. Apparently the TAG App Delivery is conducting a very interesting initiative:
- URL
- URL

That discussion goes way beyond the scope of this issue (purely focused on our mid-term survey+interview strategy) but is good food for thought on how to adopt a product-thinking mindset. There's a lot of material that can be helpful to us. They also also investigating tools and practices to run user interviews with a repeatable framework and in compliance to GDPR policies. So I'll definitely reach out, maybe they can help us

@StefanoFioravanzo thanks for kicking this off. As need steps, we might create the 1st set of survey questions completed. As we discussed, we have reused the questions from previous years, but we don't have to this year. As the same time, we need to identify users and ask if they will participate in some interviews. I think those requests need to be very defined in terms of privacy, time commitment and benefits (for the user). I would suggest defining 10 questions. Note some of those questions will be multiple choice and so we will need answers for each question. Would this be created and delivered via google forms ?

Amazing, we also have to get this into the hands of internal company employees this time. E.g. DHL, some hospitals I know of and other end user companies. Just announcing it on the internet is not enough. We also have to do so in companies intranets and private channels.

@juliusvonkohout yeah that's a whole discussion we should have - what is the "hidden" user base, how do we reach out, how do we keep track of gatekeepers, etc. I am sure these would provide a level of insight into real world production scenarios we, as a community, have not heard yet.

I have a long adopter list now and there is the KOC.
/close
Please reopen if needed.

Platform-as-a-Product Research
### Motivation

As we have learned in the Platform Engineering Maturity Model seeing your developer platform as a product is a significant sign of the maturity of your platform. But how does product management actually work with a developer platform and what tools and patterns are successfully used by platform leaders and builders?

We have a lot of assumptions around those questions, but in order to deliver an helpful guide that empowers other organisations to elevate their platform maturity by using proven product management patterns and tools, we have to put in some research and prove - or disprove - our assumptions.

### Work Issues

// TODO: Create Issues for the following topics once addressed
// TODO: Enrich this list with more topics we need to work on

* [ ] Interview Script and Guidelines
* [ ] Interview and Research Tools
* [ ] Recruiting of Interviewees
* [ ] Analyse of first set of interviews
* [ ] Publication of first results

**Hint:** Our work progress so far can be found in this research document.

### Outcome

As a first goal of this research initiative we first want to learn.
We want to learn what tools and patterns successful platform leader and builder use to manage their platform as a product.
We want to learn what potential challenges other organisations might face in their platform engineering maturity journey when productizing their platforms and how they can overcome them.
We want to learn, (if and) why successful platform teams productised their platform and what opportunities lie within introducing product thinking in other organisations.

As a second goal of this research initiative we want to share what we learned:
This can be in the form of another WG Platforms Whitepaper, Best Practice Guides, Blog Posts, Talks or any other contribution that might empower other organisations to follow in the footsteps of leading Platform-as-a-Product success stories.

Quick one @Rotfuks, URL do you think we need both of the issues still? I guess the other is more about a whitepaper than the research, but did want to point out...

Hmm good point @loujaybee - that ticket still seems to be pretty valid for what we're doing there. And I even commented there, so I should know it! :laughing:

This one was the outcome of a discussion around the next steps for the initiative and how to formalise and make the work more transparent in the scope of our working group.
The other ticket sure has way more text to go through - but it also with that has all the context and the action items that we're listing here. What do you think @mluskind, should we keep this "smaller" epic here or use the original whitepaper one?

Just talked to @mluskind, yes makes sense to close this as a duplicate and put some effort in the original ticket where people might already watch for updates. Thanks for the heads up!
And sorry @abangser for closing it while you just labeled it :heart: :laughing:

After a short checkin with @mluskind and looking at the plan of the couple of new issues we're creating here we decided it would be a better approach to use this as a dedicated ticket for the now streamlined effort that we're aiming for in this research and close the other here linked epic with the outcome of forming the initial platform-as-a-product research group and setting the topic and scope of the research we will conduct within the scope of this epic here now. Soooo, reopening it :)


Question: Dealing with words with multiple meanings?

OK, it turned out to be a lot easier than I thought to make a temporary fix: for now, there is no distinction between adjectives and nouns; e.g. the term "diagonal matrix" is simply its own entry, separate from "matrix". This should solve this problem.

Dealing with words with multiple meanings
In math, a "lattice" can refer to (at least) two completely different concepts (think "subgroup lattice" versus "$\mathbb{Z}^n$ is a lattice in $\mathbb{R}^n$"). In German, these two concepts have different words ("Verband" and "Gitter").

Can this be handled by your setup? How would I enter it?

the way this is handled is that every concept corresponds exactly to an entry on wikidata (almost always, modulo a few edge cases, and these will eventually have wikidata tags too), so for example lattice in the former sense is Q595364 and in the latter sense it is Q1096885 — the fact that they happen to have the same translation into English is then something that the database doesn't care about :-)

however, at the moment there is no entry for lattice in either sense, and there is no way for users to add entries via the submission tool, but I'm hoping to do a massive restructure of the entire project soon (see the comments that I'm just about to write on URL that will fix this

Representing compound words
In German, often words are combined to form new words. As a result, the translation of "Fourier series" is "Fourierreihe" (and *not* "Fourier Reihe"). For "power series" it is "Potenzreihe" (and "Potenz Reihe" would be *really* strange to read).

Is there a way to specify this yet? If not, perhaps one could be added?

There are also examples where it is possible to use an "adjective noun" combo, but it is unusual and the compound would be preferable: e.g. "diagonal matrix" could be translated as "diagonale Matrix", but this would be odd, the usual translation is "Diagonalmatrix".

Here's another complication: "value" is "Wert" in German. But "absolute value" is "Betrag", or sometimes "Absolutbetrag", but never "absoluter Wert" or anything else involving "Wert".

Great questions, and first of all thank you for your interest in contributing!

As for an actual answer, it basically boils down to "with how poorly the database is currently structured, this is impossible" *but* I'm hoping to get do a complete restructure over the next week or so (if I find the time) which would solve this issue entirely — I'll post on this thread when that's done!

Oh, that's great!

Unfortunately it also means I have to redo this PR I was preparing to add a bunch of German translation... :-(. But I'll manage, no worries ;-)


Question: data.table spark/databases interface?

another way to contribute, even without knowing much about how data.table works (in C or otherwise), is to look at the open issues, and try to see if you can reproduce a bug report, then add a comment on the issue that explains what you did and whether or not the issue is reproducible. (and if you can make a simpler example than what is reported, that is even better)

data.table spark/databases interface
data.table is awesome but most people don't have 100GB memory in order to handle really large data sets in memory.

Big progress has been made making the Apache Spark framework available through R in the last couple of years. Two such projects are Apache's sparkr and Rstudio's sparklyr. Both of these provide a dplyr style interface to spark's data processing engine.

As a heavy data.table user it would be amazing if there were to be a data.table interface for spark. That would make it incredibly easy for data scientists to migrate their projects from the smaller CSV style data sets to the huge data sets that can be processed by spark.

A classic data pipeline for me is
1. Bring the data into R by CSV
2. Do some pre-processing (filters, joins, aggregation, feature extraction) of the data using data.table
3. Build a model using one of R's many machine learning packages

I want to be able to migrate this to
1. Connect to data on hadoop cluster
2. Do some pre-processing (filters, joins, aggregation, feature extraction) of the data using data.table's spark interface
3. Build a model using one of spark's many machine learning algorithm.

Proposed `dplyr.table` does not need to interact with data.table at all and can be completely standalone package.
All it needs is to mimic data.table's API:

r
DT[subset, select, groupby]

That makes it much easier to deliver rather than trying to fit translation inside `[.data.table`.

I might be missing something (sorry: long, old thread with several hidden replies). But since we're ultimately talking syntax masking/mimicking, wouldn't it be easier in the long-run to create something like a `database.table` package that translates the `[i, j, by]` into the appropriate backend(s)? Or, perhaps easier for conversion, going through the new `DT(i, j, by)` functional syntax that Matt introduced not so long ago.

Having a dedicated `database.table` frontend package that directly controls the syntax generics is probably more aligned with the data.table way than going through d(b)plyr in the end.

(I think that this might Jan's point, so again apologies if I'm just quibbling over the name.)

+1 on DuckDB fwiw.

Integration of various sources/targets is a lot of dev and maintenance, therefore doing single integration to dplyr, and via it having another backends feels like much more likely to be achieved. If we want to target only spark, or only duckdb, then I agree it's better to translate directly rather than via dplyr.

An off topic: DT() has been pulled back for the moment from exported API.

I agree that it would be possible and preferable to implement this in a separate package, which hopefully would get the seal of approval #5723

also based on the new governance this is out of scope -- "Functionality that is out of current scope...Manipulating out-of-memory data, e.g. data stored on disk or remote SQL DB, (as opposed e.g. to sqldf / dbplyr)" and consensus seems to be that this should be implemented in another package, so I am closing this issue. (feel free to re-open if I have mis-understood)

one more advantage of the data.table syntax/API for arrow (R package)? I naively URL

Building a data.table community of packages with “Seal of Approval”
With the goal of building a community of packages that have similar philosophies and syntax that are separate from data.table (and outside of data.table scope #5722), we would like to set up a “Seal of Approval” (play on the mascots of `data.table`) process. The process for a package receiving the Seal of Approval could be:

* The package developer submits a standard application, indicating that their package is on CRAN, is actively maintained, and has at least one of the features listed below.
* A small number of volunteers from the data.table community review the application to ensure that the criteria are met.  (This includes no testing or vetting of the package functionality itself; only checking that it's an appropriate choice for the Seal.)
* There is some variety of public community vote to approve the package.

Approval will include being listed as a Seal of Approval package on the data.table repository and an SVG of the “seal” that they can include on their own repository/package logo. The initial idea would be packages that do at least one of the following:

* Built on the same principles as data.table (#5693)
* Extends the functionality of data.table to other contexts (e.g., to databases) with similar syntax
* Uses data.table on the backend of the package

Possible examples of this could include packages that have few dependencies (e.g. `tinytest`), extend functionality (e.g. `dtplyr`, `tidytable`, `tidyfast`), and packages that use data.table on the backend (e.g. `modelsummary`).

This process would hopefully help other developers feel more connected to data.table and be more likely to want to support it. Things for us to decide on are:

1. Does this idea resonate with the data.table community?
3. If so, does the list of criteria make sense for the purposes of the Seal of Approval?
5. What else should be considered in designing the Seal of Approval?

I believe data.table was made to play nicely with any package, by following many conventions from base R.
Making a "seal approved" may give an impression that some packages works better with data.table, while others don't work well or don't work at all with data.table...

Rather than having community of packages I would prefer to have all packages to be in a community.

Thanks for your feedback on it. I agree that data.table is nicely designed to work well with all sorts of packages (and in ways that are not always obvious!). I don't think our intention would be to say there are certain packages that work best with data.table and others don't. The goal would be to help build the community around data.table. This was just one idea of how we could engage more R users and get them into the data.table repository more. We would also hope that it would spawn more ideas of how to use data.table with other packages, across more situations. The documentation (and other resources) are vast on data.table but I think there is still a lot of users that don't find it (and how to use it) early enough.

Do you have other suggestions on how to make entry into data.table use and development easier?

I have been using data.table as a user, but I know that many parts have been written in C. I have no clue where I could start to learn C to make meaningful contributions.

I would like to have some guidelines for novice users who want to contribute to data.table.

Is there anything besides an approval process {data.table} maintainers would be committing to as part of this?

Would the approval be granted in perpetuity / renewed regularly / granted with possible revocation under "certain circumstances" (which)?

I was on the same boat, reading code of PRs was quite useful, but what was game changer is to start coding, I started with rolling mean. Then I received great feedback in my PRs, mostly from Matt, so it was easy to pick up good practices. Often I draft naive version in R, to reflect how I will code it in C, I might even skip using functions like `sum` and code it as a for loop.

Source of the project you are going to contribute to.
And which function... The one that doesn't exist yet :)

@AngelFelizR The r-contributors slack (r-contributors.slack.com) hosted a book club on learning C for R users last year:

 URL

I believe there are videos still available; try asking in the `#book-club-modern-c` channel there. Otherwise expressing new interest is a way to get the book club running a second time (others have already inquired).

As for data.table's own C code, I think the most straightforward stuff would be:

 - cj.c
 - coalesce.c
 - fifelse.c
 - idatetime.c
 - nafill.c
 - rbindlist.c
 - transpose.c

@MichaelChirico, thank you for your advice. I hope to contribute C code in the long term to continue progressing this amazing project.

I want to be prepared to the point where we can task moving data.table to work with data on disk.

I am here because the data.table survey asked if I wanted to contribute, so I started reading the issues.

To make this a concrete proposal:
1. we will add a section in README.md entitled "Seal of Approval" with a brief statement explaining that this is a list of packages which are built using data.table, and what functionality they provide.
2. the approval process is the same as for anything. If you want to list your package under Seal of Approval, submit a PR that changes README.md (add your package to the list). Make sure the change has a link to your package web page, and also a brief description of how data.table is used, and what new/interesting/unique functionality your package provides.
3. The data.table maintainers will judge submissions based on relevance -- does this package provide a new/interesting/unique functionality beyond what is provided by data.table? Also I believe Seal of Approval packages should be outside of data.table scope. A data.table maintainer will merge the PR if there is consensus, just like for any other PR.

For example I have been developing URL which provides named capture regex functionality, using and outputting data.tables, and I would like that package for inclusion under Seal of Approval.

Another example would be the mlr3 packages which are built using data.table.

I see the Seal of Approval as a way of building community, by increasing awareness about how widely-used data.table is among other R packages.

I think this will ultimately be a pretty low lift while allowing more public connections to the community.

glad to see some positive feedback to my proposal.
also would be cool to have some logo with a sea lion giving a thumbs up, does anybody have graphics/art skills? @Maradestefanis ?
My vision is that the README.md should have a one-line mention of the package, with a link to a blog post on URL which gives further details. So that would entail a little extra work for the package author: writing that blog post. (But no extra work for data.table devs, who just review the PR with a change to README.md)

Hey @tdhock I'm stepping in and I can give the logo a shot. It would be great to have it in high definition, if possible. Can you provide that? Also, is there anything else you need from me for the blog?

Hi Mara the existing logo graphics files are in URL is that high enough definition?

Yes, awesome! I've been trying it out these days

Mara Destefanis
Lic. Comunicación Social. Máster Ciencia de Datos
Tel: (+598) 99041531
Ln: URL
web: URL

El mar, 2 abr 2024 a las 18:01, Toby Dylan Hocking ( ) escribió:

Toby, Iam working on this graphics a sea lion giving a thumbs up, for now
it is regular and not really nice the result. I will try again and send you
the result in a few days.

I´ll keep pushing forward

Mara Destefanis
Lic. Comunicación Social. Máster Ciencia de Datos
Tel: (+598) 99041531
Ln: URL
web: URL

El mié, 3 abr 2024 a las 6:51, Mara Destefanis ***@***.***>)
escribió:

Hi all,

I'm reaching out with a couple ideas/options for the Seal of Approval process, to see if we can find one that everyone agrees on.

**On this repository:**

@jangorecki expressed some concern with a list on this repository's ReadMe, because it implies some kind of closed community, instead of data.table being accessible to anyone.

I propose that we simply at a `Seal-of-Approval.md` file in this repo that contains a simple list of packages that have gotten approval. Then, we can link to this md at the bottom the ReadMe, in the Community section, and reserve all additional details for blog posts on the raft instead of them clogging up this repo.

**Approval process:**

At first, @TysonStanley had suggested that approval was initialized with a PR to this repo, and @MichaelChirico was wondering about the expectations from maintainers for reviewing.

I want to suggest a reverse-order:
* Folks seeking Seal of Approval would submit by PR to The Raft.
* Here is a sample draft for what would be submitted.
* I would act as a first screen, and then if the proposal seems reasonable, I would PR this repo with the relevant information.
* I could also, if desired, tag reviewer(s) in my PR.

This would be kind of a mini "journal-style" process that would maybe take some of the burden off the maintainers.

**Longevity:**

Michael also asked about whether this approval is granted in perpetuity or not. I think just workload wise, we wouldn't commit to periodic re-reviews. However, if someone were to alert us to an issue with a package - say, it's no longer actively maintained - we'd take it off the list at maintainer's discretion.

**Type of SoA Packages:**

I've come up with four types of packages that might merit approval; in principle, a submitter would have to justify the package falling in one or more of these categories. I'd love feedback if anything seems amiss:

- [ ] **An extension package:** Adds to the internal functionality of `data.table`

- [ ] **An application package:** Uses `data.table` to accomplish a particular task or analysis.

- [ ] **A bridge package:** Translates `data.table` syntax to different syntax or provides helper functions for transitioning between `data.table` and another object type.

- [ ] **A partner package:** Not necessarily directly connected to `data.table`, but deliberately follows the core philosophies of `data.table`.

---

So, tl;dr, in this proposal:
* Someone submits info on their package to The Raft
* I review it, then PR this repo adding the package name to a simple .md list, and supply relevant info for maintainters
* Maintainers merge PR at their discretion
* Package is then published as a blog post to The Raft

Let me know if this sounds workable to you, or if you have other suggestions! :)

Since there are no major blocking concerns with Kelly's most recent proposal, I would suggest that we go ahead with that.

Since the seal of approval is moving forward with Kelly's suggestion, should we close this now?


Question: OBS Studio not working?

The first way is the correct way. What is your wayfire.ino file? The example config contains an autostart entry for exporting the necessary environmental variables to dbus/systemd.

OBS Studio not working
*(Sorry I deleted the template, but I made sure to include all the information that it asked)*

Obs studio doesn't propose any screen output source.

The reason I ask that here and not in obs-studio repository is because obs works in sway, I switched from sway to wayfire recently and this is the only inconvenience I noticed. I believe Obs uses something called pipewire in wayland environments.

**versions**
wayfire 0.7.4
obs 29.1.1 (64bit)

Here's the log when I run Obs from the command line, maybe it can help to debug:

debug: Found portal inhibitor
debug: Attempted path: share/obs/obs-studio/locale/en-US.ini
debug: Attempted path: /usr/share/obs/obs-studio/locale/en-US.ini
debug: Attempted path: share/obs/obs-studio/locale.ini
debug: Attempted path: /usr/share/obs/obs-studio/locale.ini
debug: Attempted path: share/obs/obs-studio/themes/Yami.qss
debug: Attempted path: /usr/share/obs/obs-studio/themes/Yami.qss
info: Platform: Wayland
info: CPU Name: AMD Ryzen 9 5900HS with Radeon Graphics
info: CPU Speed: 4466.433MHz
info: Physical Cores: 8, Logical Cores: 16
info: Physical Memory: 15381MB Total, 2761MB Free
info: Kernel Version: Linux 6.1.18-200.fc37.x86_64
info: Distribution: "Fedora Linux" 37
info: Desktop Environment: Wayfire (wayfire)
info: Session Type: wayland
info: Qt Version: 6.4.3 (runtime), 6.4.3 (compiled)
info: Portable mode: false
info: OBS 29.1.1 (linux)
info: ---------------------------------
info: ---------------------------------
info: audio settings reset:
 samples per sec: 48000
 speakers: 2
 max buffering: 960 milliseconds
 buffering type: dynamically increasing
info: ---------------------------------
info: Initializing OpenGL...
info: Using EGL/Wayland
info: Initialized EGL 1.5
info: Loading up OpenGL on adapter AMD AMD Radeon Graphics (renoir, LLVM 15.0.7, DRM 3.49, 6.1.18-200.fc37.x86_64)
info: OpenGL loaded successfully, version 4.6 (Core Profile) Mesa 23.0.3, shading language 4.60
info: ---------------------------------
info: video settings reset:
 base resolution: 352x397
 output resolution: 352x396
 downscale filter: Bicubic
 fps: 60/1
 format: NV12
 YUV mode: Rec. 709/Partial
info: NV12 texture support not available
info: P010 texture support not available
info: Audio monitoring device:
 name: Default
 id: default
info: ---------------------------------
warning: Failed to load 'en-US' text for module: 'decklink-captions.so'
warning: Failed to load 'en-US' text for module: 'decklink-output-ui.so'
libDeckLinkAPI.so: cannot open shared object file: No such file or directory
warning: A DeckLink iterator could not be created. The DeckLink drivers may not be installed
warning: Failed to initialize module 'decklink.so'
info: [pipewire] No captures available
error: os_dlopen(libnvidia-encode.so.1->libnvidia-encode.so.1): libnvidia-encode.so.1: cannot open shared object file: No such file or directory

nvc0_screen_create:1169 - Error allocating PGRAPH context for M2MF: -110
error: VAAPI: Failed to initialize display in vaapi_device_h264_supported
info: VAAPI: API version 1.16
info: FFmpeg VAAPI H264 encoding not supported
nvc0_screen_create:1169 - Error allocating PGRAPH context for M2MF: -16
error: VAAPI: Failed to initialize display in vaapi_device_hevc_supported
info: FFmpeg VAAPI HEVC encoding not supported
info: [obs-websocket] [obs_module_load] you can haz websockets (Version: 5.2.2 | RPC Version: 1)
info: [obs-websocket] [obs_module_load] Qt version (compile-time): 6.4.3 | Qt version (run-time): 6.4.3
info: [obs-websocket] [obs_module_load] Linked ASIO Version: 102600
info: [obs-websocket] [obs_module_load] Module loaded.
info: [vlc-video]: VLC 3.0.19 Vetinari found, VLC video source enabled
info: ---------------------------------
info: Loaded Modules:
info: vlc-video.so
info: text-freetype2.so
info: rtmp-services.so
info: obs-x264.so
info: obs-websocket.so
info: obs-vst.so
info: obs-transitions.so
info: obs-outputs.so
info: obs-libfdk.so
info: obs-filters.so
info: obs-ffmpeg.so
info: linux-v4l2.so
info: linux-pulseaudio.so
info: linux-pipewire.so
info: linux-jack.so
info: linux-capture.so
info: linux-alsa.so
info: image-source.so
info: frontend-tools.so
info: decklink-output-ui.so
info: decklink-captions.so
info: ---------------------------------
QWidget::setTabOrder: 'first' and 'second' must be in the same window
info: ==== Startup complete ===============================================
info: All scene data cleared
info: ------------------------------------------------
info: [Python: resize-base-resolution-from-source2.py]
info: [obs-scripting]: Loaded python script: resize-base-resolution-from-source2.py
info: pulse-input: Server name: 'PulseAudio (on PipeWire 0.3.70) 15.0.0'
info: pulse-input: Audio format: s16le, 48000 Hz, 2 channels
info: pulse-input: Started recording from 'bluez_output.58_FC_C6_1C_55_4E.1.monitor' (default)
info: [Loaded global audio device]: 'Desktop Audio'
info: pulse-input: Server name: 'PulseAudio (on PipeWire 0.3.70) 15.0.0'
info: pulse-input: Audio format: s32le, 48000 Hz, 2 channels
info: pulse-input: Started recording from 'alsa_input.pci-0000_08_00.6.analog-stereo' (default)
info: [Loaded global audio device]: 'Mic/Aux'
error: Source ID 'pipewire-desktop-capture-source' not found
error: Failed to create source 'Screen Capture (PipeWire) 2'!
info: Switched to scene 'asdf'
info: ------------------------------------------------
info: Loaded scenes:
info: - scene 'asdf':
info: - scene 'sdf':
info: - source: 'Screen Capture (PipeWire) 2' (pipewire-desktop-capture-source)
info: ------------------------------------------------
info: adding 21 milliseconds of audio buffering, total audio buffering is now 21 milliseconds (source: Desktop Audio)

@ammen99 I found this thread URL and decided to include

[autostart]
01_xdp=/usr/libexec/xdg-desktop-portal
02_xdpg=/usr/libexec/xdg-desktop-portal-gtk
03_xdpw=/usr/libexec/xdg-desktop-portal-wlr

in my config file.

It seems that it works now! though I really don't know why. Does it change the way things are getting rendered on the screen in general?

(If there is a section in the documentation that talks about this, could you please redirect me?)

So far wayfire is really boosting my production and I can't be thankful enough for that, if there is anyway we can make donations, please let me know.

Glad to hear it works :)

For screen recording/sharing/etc. Wayfire and most wlroots-based compositors (Sway too) have a semi-private protocol, called wlr-screencopy. However, after we had this protocol, another way for this was standardized, which is also supported by GNOME and KDE (via xdg-desktop-portal). There is `xdg-desktop-portal-wlr` which basically translates from the wlroots' protocol to the xdp one, and OBS will not work unless this bridging component runs in the background (since OBS uses the standard xdp recording protocol).

@ammen99 Well... I think it is somehow related but do you happen to know why file picker doesn't show up? For instance when I click "upload a file" in browser, a window should open so I can choose a file on my local machine, but for some reason it doesn't show up.

I thought autorun the portals would fix this issue but it doesn't.

Do you run xdg-desktop-portal-gtk? It should be autostarted if you have the dbus/systemd env vars exported by autostart, the wiki has a section on that iirc.

@ammen99 yes here's the `[autostart]` section in my config file

[autostart]
autostart_wf_shell = true
00_dbus = dbus-launch
01_xdp=/usr/libexec/xdg-desktop-portal
02_xdpg=/usr/libexec/xdg-desktop-portal-gtk
03_xdpw=/usr/libexec/xdg-desktop-portal-wlr
portal=/usr/libexec/xdg-desktop-portal-gtk

Maybe I shouldn't use all the portals at once?

Also I have the problem in chrome right now, but file chooser seems to work in Firefox. Maybe I should open a new issue for that?

Here's my config (no need for a new issue, this isn't a Wayfire issue anyway, it is all about the dbus stuff):

[autostart]
0_0 = systemctl --user import-environment
portal = sleep 1 && (XDG_SESSION_TYPE=wayland XDG_CURRENT_DESKTOP=sway /usr/libexec/xdg-desktop-portal --replace & /usr/libexec/xdg-desktop-portal-wlr)

You can either set `XDG_*` before starting Wayfire, or replace the already-running portals like I do here.

You do need the systemctl command, or at least `dbus-update-activation-environment WAYLAND_DISPLAY DISPLAY XAUTHORITY` for all dbus-activated stuff to work, that includes file chooser dialogs.

Having same issue, I launched `xdg-desktop-portal-wlr`, restarted `pipewire`, but I still get the `info: [pipewire] No captures available` message on `obs` launch, and there are no relevant outputs in `helvum`.

To have the output picker show up, you need to have `slurp` installed.

Really strange that it's not showing the Screen Capture (Pipewire) with the XDG Desktop Portal loaded.

Coming back to this issue, trying to find "window capture" (not "screen capture") but I can't find it, I don't know if that's a limitation of obs on wayland or another dbus issue.
Can you guys tell me if you have it?

If I understand correctly, it's a limitation of wayland. There is currently, no way to identify a window. This is likely to change in the future, with the introduction of ext-foreign-toplevel-list protocol which provides a means to identify views with a unique id. This isn't yet supported by wlroots. Once the support for this is added, then you'll be able to capture views.

I think this might be the case of wlroots and wayfire supporting this protocol, which outlines the ability to capture single toplevels.

What's the right way to start a wayland session?
there'are 2 ways to start a WM session for sway/wayfire/...
no matter start the WM from tty manually, or let DM load the session for WM, or start a command by DM.
1. run sway/wayfire directly.
2. run sway/wayfire by dbus: `dbus-launch wayfire`

i am using greetd and tui-greeter.
## Way 1, launch WM directly
all things works but except gvfs and fcitx5-rime.
- gvfs fails
thunar report the gvfs was not work currectly, and there's no vols appear in left panel of thunar.
for tracering the issue, all of them report an GLIB error.
for example , this is the report of gvfs debug , using comomand GVFS_DEBUG=1 $(find /usr/lib* -name gvfsd 2>/dev/null) --replace 2>&1 | tee gvfsd.log :

(process:14320): GLib-GObject-CRITICAL **: 11:21:21.921: g_object_new_with_properties: assertion 'G_TYPE_IS_OBJECT (object_type)' failed

(process:14320): GLib-GIO-CRITICAL **: 11:21:21.924: g_volume_monitor_get_mounts: assertion 'G_IS_VOLUME_MONITOR (volume_monitor)' failed

(process:14320): GLib-GObject-WARNING **: 11:21:21.924: invalid (NULL) pointer instance

(process:14320): GLib-GObject-CRITICAL **: 11:21:21.924: g_signal_connect_data: assertion 'G_TYPE_CHECK_INSTANCE (instance)' failed

(process:14320): GLib-GObject-WARNING **: 11:21:21.924: invalid (NULL) pointer instance

(process:14320): GLib-GObject-CRITICAL **: 11:21:21.924: g_signal_connect_data: assertion 'G_TYPE_CHECK_INSTANCE (instance)' failed

- fcitx5-rime
this input method not work in chromium.

- obs-studio works in this way.

## Way 2, lauch WM by dbus

if i use this command to let the DM start the WM

cmd='tuigreet -cmd dbus-launch wayfire'

or type ` dbus-launch wayfire` in tty

- gvfs works.

- fcitx5-rime works everywhere.

- but obs-studio can't take screen record (no `capture screen source (pipewire)` appears), cant' start virtual camera.

error: Failed to inhibit: GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: on path org.freedesktop.portal.Inhibit object not found“/org/freedesktop/portal/desktop”interface

- some program fails, like celluloid
can't be started by GUI, both in thunar or wofi.
but can be started by terminal, with the same error reported above.
all packages like xdg-desktop-portal , xdg-desktop-portal-wlr already installed.

## additional

i'm using void-linux.
dbus-daemon was runing in all these situation.
dbus service was enabled and elogind was launched by greetd.

i let greetd to start a script called wayfire-run :
in this script, i exported some enviroment and start wayfire:

bash

# cat $(which wayfire-run)

#!/bin/sh

# Session
export XDG_SESSION_TYPE=wayland
export XDG_SESSION_DESKTOP=Wayfire
export XDG_CURRENT_DESKTOP=Wayfire

# Wayland stuff
export MOZ_ENABLE_WAYLAND=1
export QT_QPA_PLATFORM=wayland
export SDL_VIDEODRIVER=wayland
export _JAVA_AWT_WM_NONREPARENTING=1

# exec dbus-launch --exit-with-session wayfire $@
exec wayfire $@

dbus service was enabled, so am i still need to start a dbus-launch on start ?

shell
# ls /var/service/
NetworkManager@ agetty-tty3@ bluetoothd@ polkitd@
acpid@ agetty-tty4@ cupsd@ rtkit@
agetty-tty1@ agetty-tty5@ dbus@ udevd@
agetty-tty2@ agetty-tty6@ greetd@ uuidd@

i have tried `dbus-run-session wayfire`, but it results the same situations as dbus-launch.

when runing session by `dbus-run-session`

# ps -aux|grep dbus

root 1103 0.0 0.0 2352 1160 ? Ss 18:35 0:00 runsvdir -P /run/runit/runsvdir/current log: pid=1135 comm="NetworkManager -n") (using servicehelper) dbus-daemon[1127]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher' dbus-daemon[1127]: [system] Activating service name='fi.w1.wpa_supplicant1' requested by ':1.7' (uid=0 pid=1135 comm="NetworkManager -n") (using servicehelper) dbus-daemon[1127]: [system] Successfully activated service 'fi.w1.wpa_supplicant1' ...
root 1113 0.0 0.0 2200 764 ? Ss 18:35 0:00 runsv dbus
dbus 1127 0.0 0.0 3720 2900 ? S 18:35 0:00 dbus-daemon --system --nofork --nopidfile
shvo 1397 0.0 0.0 3780 2224 ? Ss 18:35 0:00 /usr/bin/dbus-daemon --syslog --fork --print-pid 4 --print-address 8 --session
shvo 1496 0.0 0.0 3276 2440 ? S 18:35 0:00 /usr/bin/dbus-daemon --config-file=/usr/share/defaults/at-spi2/accessibility.conf --nofork --print-address 11 --address=unix:path=/run/user/1000/at-spi/bus

while open chromium or electron , every works , but report:

no such interface : org.freedesktop.portal.FileChooser

file chooser works.

Ensure `xdg-desktop-portal` is running. If you want to start it, add it to autostart

[autostart]
01_xdp=/usr/libexec/xdg-desktop-portal
02_xdpg=/usr/libexec/xdg-desktop-portal-gtk
03_xdpw=/usr/libexec/xdg-desktop-portal-wlr

`/usr/libexec/xdg-desktop-portal` is runing by runit.

02_xdpg=/usr/libexec/xdg-desktop-portal-gtk
03_xdpw=/usr/libexec/xdg-desktop-portal-wlr

i run these two manually, but nothing change.

@superiums You are not meant to run xdg-desktop-portal-gtk manually. Can you post your full wayfire.ini file?

My uwsm might be of interest
dbus session stuff should be handled on systemd/logind side in general.

yeh, it's an session stuff and should be handled by logind, i've seen uwsm,
but void linux has no systemd, services was loaded by runit. does it work in this situation ?

Then no, sorry. It relies on systemd unit mechanisms heavily.


Question: NavigationBar onTap ripple alignment bug?

cc: @HansMuller
Not sure why the ink ripple is shifted in M3. It might be due to the different label sizes in M2 and M3.
I will fix this with a better a rect.

NavigationBar onTap ripple alignment bug
Hi, using Flutter 3.7 with `flex_color_scheme: ^6.1.2`.

flex

dart
FlexThemeData.dark(
 scheme: FlexScheme.flutterDash,
 surfaceMode: FlexSurfaceMode.levelSurfacesLowScaffold,
 blendLevel: 15,
 subThemesData: const FlexSubThemesData(),
 keyColors: const FlexKeyColors(),
 visualDensity: FlexColorScheme.comfortablePlatformDensity,
 useMaterial3: true,
 swapLegacyOnMaterial3: true,
)

Thanks for your hard work!

Thanks @esenmx for the issue report.

Yes I am aware of this issue. It is a bug in Flutter 3.7, basically a regression, a bug that does not exist in Flutter 3.3 nor in Flutter master channel, since there is fix for it already in master channel.

Unfortunately that fix did not make it into Flutter 3.7 stable as cherry pick for the release. It would have needed an extra cherry pick for the 3.7 stable release, since 3.7 release had already had its code freeze when the fix was made. Now we have to wait and hope a fix for this is included in the first Flutter 3.7 hotfix (3.7.1).

The ink bug is only visible if the `NavigationBar` height is different than the default M3 height of 80dp. If you make the height lower than 80dp, the ink effect moves the diff amount from 80dp below it, if you make the height higher than 80dp, the effect moves up with same amount that it is higher than 80dp. So there is some kind of offset error on the ink ripple.

FlexColorScheme defaults to an opinionated design that is only 62 dp heighh in both M2 and M3. As a workaround, if you set its height to 80dp

dart
 subThemesData: const FlexSubThemesData(
 navigationBarHeight: 80.0,
 ),

The issue becomes invisible.

I will when I release the FCS 7.0 stable version introduce a with FCS 6.x style break, that sets the default when opting in on M3 to 80dp instead of 62dp. Which as said is the default for M3 design anyway. At least then this issue is not visible by default when opting in on M3. It will still be there when using M2 though, since I don't want to break past default for FCS when using M2. I will leave the manual workaround to set it to 80dp to be the choice of the developer when using M2. Well let's see, it depends a bit on the timeline for the fix to land.

The PR that fixes this issue can be found here:
 URL

The author of the fix @TahaTesser and @Piinks are aware that this might be a big enough issue to be included in the first hotfix to Flutter 3.7.

I will keep this issue open until there is a fix for it Flutter. I will also add it to known issues in the docs.

@esenmx for information the fix to this Flutter 3.7 bug has been cherry-picked here URL it will thus most likely be included in the first Flutter hotfix release 3.7.1.

Sadly it did not make it to 3.7.1, maybe 3.7.2 then. Yes I know, quite disappointing, but I have no control over what lands when in the framework :smile:

@esenmx the Flutter SDK fix for this landed in Flutter hotfix 3.7.2. Closing this issue.
 URL

[NavigationBar] ink ripple positioned is shifted in M3
At first, I thought it might be a resolution issue it looks like the ink ripple rendered at a different vertical offset in M3 for some reason.

 URL

When running the example as it is without enabling M3, ink is rendered at the right location and you can't see peeking outside but when M3 is enabled, it's slightly off.

Screenshot 2022-12-21 at 01 15 1

Screenshot 2022-12-21 at 01 12

When you change the height from 80dp, the M3 default, the ink ripple shifts with offset amount up/down equal to height change diff and direction. It affects both M2 and M3 mode. The issue did not exist in stable 3.3 but has now regressed into stable 3.7.

See example here: URL

Sadly it did not make it into 3.7.1, nor a bunch of other important M3 fixes that exist in master, oh well, later then... 😄


Question: Regression in v2: parent() returns old data even after the parent was invalidated using invalidate()?

The original issue is about `await goto(..); await invalidate(..)`, which calls `load` twice which is expected behavior: In the linked reproduction, the `goto` makes `load` rerun because of the changed search parameter and the `invalidate` afterwards makes it run again because the dependency it has declared with `depends` is invalidated.

The issue some others in this thread have about `await invalidate(..); await goto(..)` calling it twice was a bug that has been fixed by #11268

Regression in v2: parent() returns old data even after the parent was invalidated using invalidate()
### Describe the bug

This is hard to explain, but I'll do my best.
Imagine the following route structure:

routes/
├─ +page.svelte
├─ +layout.ts
├─ [id]/
│ ├─ +page.svelte
│ ├─ +page.ts
├─ add/
│ ├─ +page.svelte

The root page shows a basket of fruits, which is loaded from an API in the root +layout.ts. It also includes a button that links to the add page.
The add page includes a button that adds fruit to the basket by posting to the API. After doing so, it invalidates the root +layout.ts load function and then navigates to the root page.
The [id] page only verifies whether a fruit with such an ID exists in the basket. It does that using the +page.ts load function, which fetches the parent using `await parent()` and then observes, whether a fruit with such an ID is in the parent data.

Now, let's imagine I do this:
- I navigate to the add page
- I press the button to add a fruit
- I get redirected to the root page, where the fruit is now listed.
- I click the fruit, which redirects me to /[id] with the fruit id.
- However, the page tells me that no such fruit exists.
- I click back and navigate to the root page, which now does not list the fruit at all.
- I refreshed the page, and now everything works.

Here's a video where I do that:

 URL

However, when I downgrade to sveltekit v1, or use `invalidateAll()` instead of `invalidate()`, everything works correctly, and the `parent()` function returns correct data.

### Reproduction

SvelteKit v2: URL
The issue here is easily reproducible as shown in the video

SvelteKit v1: URL
The issue is not reproducible, everything works as expected. See the video:

 URL

And as a bonus, SvelteKit v2 using `invalidateAll()`: URL
The issue is also not reproducible, everything works as expected.

### Logs

Shell
Not applicable

### System Info

Shell
System:
 OS: Linux 6.6 Arch Linux
 CPU: (24) x64 AMD Ryzen 9 5900X 12-Core Processor
 Memory: 35.83 GB / 62.70 GB
 Container: Yes
 Shell: 3.7.0 - /usr/bin/fish
 Binaries:
 Node: 18.18.2 - /usr/bin/node
 Yarn: 1.22.21 - /usr/bin/yarn
 npm: 10.2.5 - /usr/bin/npm
 pnpm: 8.14.0 - /usr/bin/pnpm
 bun: 1.0.18 - /usr/bin/bun
 Browsers:
 Chromium: 120.0.6099.199
 npmPackages:
 @sveltejs/adapter-auto: ^3.0.0 => 3.1.1
 @sveltejs/kit: ^2.0.0 => 2.4.1
 @sveltejs/vite-plugin-svelte: ^3.0.0 => 3.0.1
 svelte: ^4.2.7 => 4.2.9
 vite: ^5.0.3 => 5.0.12

### Severity

blocking an upgrade

### Additional Information

PS: Even though using `invalidateAll` is a workaround, I've decided to use the "blocking an upgrade" severity as rerunning all the load functions is not acceptable.
**I hope I explained the issue well, if you have any questions, ask me.**

This bug is very real but hasn't been acknowledged by the team yet, see URL

And also :

 URL
 URL
 URL
 URL
 URL
 URL

Thanks for the information! I hope they tackle this soon as this is a very severe issue for me.

Partial invalidation has never been fully functional in SvelteKit, even pre 1.0. I suspect none of the maintainers know how to fix it.

@Rich-Harris I'm sorry for mentioning you, as I'm sure you have a lot of work. But you're the author of the original PR which @gterras suspects to be the culprit. Unfortunately, the issues with invalidating have been here for almost half a year, and they seem to progressively get worse. Could you please take a look into this?

I can confirm that the PR is causing the issue, removing the `invalidated.length = 0;` does fix my issue, but it might cause a new one.

@stalkerg hey! Seeing that you liked the issue, do you have the same problem?

@kyngs not yet; I did it to improve visibility. Unfortently I have no time now to go deeper and fix it. If nobody fixes it next week, I will try to help.

I'm experiencing the same issue. In my case, I have a page that calls `parent()` and an invalidate call in the layout that redirects to `/login` when an auth check fails. Previously that worked just fine, but now it gets stuck showing the same page, but loaded with `/login`'s page data.

Is there any movement on this? This is a fairly serious regression, and is preventing us from moving to SK v2.

I did create a simple GitHub project that demonstrates the problem, if that helps... URL

Invalidate + Goto triggers load() twice with custom identifier
### Describe the bug

Hi,

It seems that using a goto() + invalidate() triggers twice an invalidate method which calls twice the load() method from +page.server. This happens only when using custom identifiers as stated by official documentation : URL

GIVEN load() depending on (foo:bar) WHEN goto + invalidate(foo:bar) THEN load method called twice (KO)
GIVEN load() depending on (foo-bar) WHEN goto + invalidate(foo-bar) THEN load method called once (OK)

If I just misunderstood this documentation, please let me know :)
Maybe invalidate + goto should be used together ? But goto doesn't seem to invalidate at every searchParams change (when modyfing existing searchParam value)
Thanks for your work !

### Reproduction

Minimal reproduction:

 URL

+page.js

 import { goto, invalidate } from "$app/navigation";

 /** @type {import('./$types').PageData} */
 export let data;

 async function onBtnClickKO () {
 await goto(`?page=${data.page +1 ?? 0}`);
 await invalidate('search:something');
 };

 async function onBtnClickOK () {
 await goto(`?page=${data.page +1 ?? 0}`);
 await invalidate('search-something');
 };

 Welcome to SvelteKit
 Visit

 Invalidate KO
 Invalidate OK

+page.server.js

export const load = async ({ url, depends }) => {
 const searchParams = new URL(url).searchParams;

 const page = +(searchParams.get('page') ?? 0);

 depends('search:something');
 depends('search-something');
 console.log('LOAD, THIS LOG SHOULD BE DISPLAYED ONLY ONCE PER REFRESH');

 return {
 page,
 results: [{ a: '1', b: '2' }]
 };
};

### Logs

_No response_

### System Info

Shell
"@sveltejs/kit": "1.22.1",

### Severity

annoyance

### Additional Information

_No response_

We just discovered the same issue in our application. Whenever we have the following code:

js
someAPI.updateEntity(entity)
 .then(async () => {
 await invalidate("app:someEntity")
 await goto("/url/of/entity")
 })

we end up with the respective load function being called twice:

Screenshot 2023-12-15 at 18 43 56

To add to the fun, if we have something like this:

js
someAPI.updateEntity(entity)
 .then(async (res) => {
 await invalidate("app:someEntity")
 await invalidate("app:listOfEntities")
 await goto("/url/of/entity")
 })

we end up with the load function that has `depends("app:someEntity")` being triggered 3 times (the `depends("app:listOfEntities"` load function is in the `+page.ts` of a parent route).

We tried that both with using URL or custom identifier, the results are the same.

I'd argue that the severity is more than "annoyance", as this slows down the page, as we have to wait for 2 or 3 repeated GET requests to finish. Also it puts unnecessary pressure on the API server, as it has to serve a lot of pointless GET requests.

Indeed in our case we have frequently invalidated load functions triggering large computations and this bug causes us trouble, is it expected to be fixed at some point ?

Great to know I am not going crazy! I was flumoxxed about this issue! #11663
Needs a fix ASAP

I can also confirm that upgrading to SvelteKit v2.0 solved the problem for us.

Careful there are chances you went from a major annoyance bug to an app breaking bug, see URL it's not obvious to see at first.


Question: Engine: `local_ident`'s `id` type is opaque?

Yeah. I see what you mean. Indeed, we may end up with a messy and complicated API...

The implementation itself is already quite messy and complicated, actually 😅 I would like to revisit it at some point...

I think we should maybe brainstorm and figure out what kind of naming transformations one could want, and then decide whether a unified API is possible. I wonder what would be the best way of doing that: maybe a GH discussion?

I think a unified API & unified transformation "engine" would be great. Having a declarative description of what a specific backend needs in terms of naming scheme, and then invoking the name transformation mechanism seems the right think to do to me. The concrete_ident module has invariants, and it's easy to break them, so doing something uniformly would be great.

Engine: `local_ident`'s `id` type is opaque
The interface for `local_ident` in the engine is defined like this:

ocaml
 type id [@@deriving show, yojson, hash, compare, sexp, eq]

 val mk_id : kind -> int -> id

 type t = { name : string; id : id }
 [@@deriving show, yojson, hash, compare, sexp, eq]

The `id` type is opaque so backends cannot inspect the `kind` of a `local_ident`.

Without this information, certain things become impossible.

Here I think I have the same concerns as #793: this kind of thing should be used for naming scheme only, what kind of things do you need for local idents?

It’s been a while since I looked at this. IIRC, we needed to inspect the kind to see if a local variable is a const generic (which we treat differently than other local vars).

Mmh, I see, looks like a legitimate use to me, I can see phases needing such info. Let's merge that PR (finally... I'm sorry --again-- for how long that took...)

Raw view on `concrete_ident`
Currently, `concrete_ident` is exported to engines as an opaque type. To get its underlying content, one would need to make a `VIEW_API` which transform the underlying path and definition into a view.

While the `VIEW_API` can accept options such as a `NAME_POLICY`, it seems intractable to have an API that is flexible to accommodate all possible name transformation schemes. For example, there is currently no way to configure the prefix scheme built-in to the default view api.

It seems appropriate to just export the raw underlying content without any transformations, giving each engine the option to have full control over how to translate `concrete_ident`s.

@W95Psp @paulmure what's the state here? It looks to me like the PR only needs a review?

Hi @paulmure, I've been trying to review a few times, but each time I end up wondering if that's the right thing to do.

I agree with you, the view API is restrictive: it imposes a common view on naming for every backend.
But I suspect most of the backend will have the same needs.
I don't want the backends to re-implement, over and over, name translation.

Could you give me some examples of a custom naming scheme you would like to have, and that would be too specific to your use case to be mainlined?
Clearly, for something like prefixes, we could make it parametric (as you say)

Controlling the prefix is definitely a priority right now. But we think it would also be useful to control how much of the fully qualified path is used based on what’s currently imported. But this may be easily implemented by allowing the view API to export a list of path items.

The reason why I suggested exporting the entire type is because it seems like this sort of fine-grained tweaking can easily lead to a messy and unprincipled view API. But there’s definitely a lot of room for design choices and I’m not entirely certain what’s the best solution here.


Question: RedisReceiver FeatureRequest: Cluster Info Capability?

This is a very interesting feature, and many of the plugins in otel col contrib are currently restricted to running on a single instance.

RedisReceiver FeatureRequest: Cluster Info Capability
### Component(s)

receiver/redis

### Is your feature request related to a problem? Please describe.

Redis Receiver is currently only for monitoring individual redis performance. Need ability to also record the cluster info related to the individual instance

### Describe the solution you'd like

Additional Optional Metrics that use the CLUSTER INFO command to return the fields returned from that command.

### Describe alternatives you've considered

Bash Script to parse the commands manually but it would be nice to have it included in the receiver

### Additional context

Example Return from CLUSTER INFO command on a single node
`cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:152
cluster_size:104
cluster_current_epoch:827
cluster_my_epoch:753
cluster_stats_messages_ping_sent:7235
cluster_stats_messages_pong_sent:7387
cluster_stats_messages_fail_sent:309
cluster_stats_messages_sent:14931
cluster_stats_messages_ping_received:7387
cluster_stats_messages_pong_received:7212
cluster_stats_messages_fail_received:3
cluster_stats_messages_received:14602
total_cluster_links_buffer_limit_exceeded:0
`

This can be more complicated than it seems. The receiver is intended to scrape metrics from individual instances. If we just add the cluster-related metrics, they will be published from all the scrapers as duplicated metrics. I believe, this should be controlled by leader election, which is currently in development for Kubernetes. The leader elector would ensure that the cluster-wise metrics are collected only from only collector instance.

cc @rakesh-garimella for visibility

If it was cluster wide metrics I would agree with you. However this is the cluster metrics according to the local instance which is not duplicated across all nodes in the cluster as it includes metrics such as messages its recieved from other nodes and what it thinks the current epoch is.

@SuprPhatAnon, thanks for clarifying. That should be straightforward to add, then. Feel free to contribute if you have a chance

@akstron I was going to pick it up in a couple weeks when things got more calm but feel free to tackle it

Hi @dmitryax , I would like to work on this issue.
Here's what I've understood from the issue description:

Current behaviour - The Receiver can scrape metrics only from a single Redis node.

Required behaviour - Support has to be added for `CLUSTER INFO` command also, so that it can also collect metrics from that command.

Have I understood this correctly?

Leader Election Extension
### The purpose and use-cases of the new component

There can be a use case where one needs to run components in an HA mode. But currently running multiple instances of components like `k8sclusterreceiver` would
cause collection of duplicate metrics. In order to be able to run such receivers in an HA mode one needs to make sure only one receiver is active. This can be achieved by utilizing the leader election mechanism provided by Kubernetes client library.

The support for extension can be implemented as an extension so that any component that wants to leverage leader election can use it easily.

### Example configuration for the component

yaml
receivers:
 k8sclusterreceiver:
 leader_elector: leader_elector
extensions:
 leader_elector:
 auth_type: kubeConfig
 lease_name: foo
 lease_namespace: default

### Telemetry data types supported

traces, metrics, and logs

### Is this a vendor-specific component?

- [ ] This is a vendor-specific component
- [ ] If this is a vendor-specific component, I am a member of the OpenTelemetry organization.
- [ ] If this is a vendor-specific component, I am proposing to contribute and support it as a representative of the vendor.

### Code Owner(s)

@skhalash @a-thaler @rakesh-garimella

### Sponsor (optional)

@dmitryax (volunteered)

### Additional context

I work at SAP on a project called Kyma: URL In Kyma, we recently developed such a receiver and we are already testing it out in a production setup: URL

We’ve noticed discussions in the community about introducing leader election in the k8sobjects and k8scluster receivers. That's why we believe that a generic mechanism could be quite beneficial. If there’s interest, we are ready to contribute it to the Open Telemetry project.

Hey @ChrsMark! Thanks for the feedback. Yes, exactly the leader is responsible for enabling a sub-receiver (such as `k8sclusterreceiver` and `k8sobjectsreceiver`) if the collector is running as a DaemonSet or a Deployment.

Yes, I fully agree with the concerns about resource limits/requests. However, as you said, it should be properly documented. Not sure if there is a way to workaround it.

Regarding putting some extra load on the k8s API - do you mean querying/updating leases?

@skhalash yes, but this is something that can also be properly documented along with some perf tests results so as users be aware of any possible impact to their clusters.

Why does it have to be a separate receiver? I think this should be an extension providing an interface that _any_ receiver can connect to check if it's a leader (have to do the work) or not (do nothing).

Hey @dmitryax! Thanks so much for your response! We’d be happy to contribute such a component. I don’t have much experience with extensions—would this mean that every receiver needing this functionality would require code modifications? Implementing it as a delegating receiver, similar to receiver-creator, could avoid it.

Modifying code is fine - better than having two similar receivers. It's better to write some code, but keep the user interface clean. Restricting it to receiver_creator, we won't be able to use leader election for receivers that don't work with receiver creator, for example URL or don't need to use it all, e.g. collecting data from a static endpoint. Also, future of receiver_creator is unclear. We might take a different approach to discovery and consolidate all scraping receivers in one, see URL

Thanks for the response! We'll explore the idea of implementing leader election as an extension instead of a receiver and will get back to you.

Hey @dmitryax,

We're exploring a specific use case and wanted to get your thoughts. The scenario involves multiple receivers within a single running collector instance that may need to operate in singleton mode, using leader election. To support this, each receiver might need its own lease, allowing each to be managed by a different leader.

We're considering two possible approaches:

1. Embedding the lease configuration directly within each interested receiver’s configuration.
2. Creating multiple instances of a new extension, like `singleton/foo` and `singleton/bar`, each referenced by its respective receiver.

The second approach seems simpler to implement. However, from what I’ve seen, there aren’t any examples in the wild of multiple instances of extensions being registered. Do you think this approach would be feasible?

We did some investigation and agree that implementing leader election as an extension is indeed a cleaner solution. We will provide a PR with the implementation soon.

Hi @chenlujjj I am actively working on this. I plan to create a PR with initial implementation of the extension in this week.

Initial Implementation for the extension has been done here. @dmitryax please have a look.

The second PR with actual implementation is here: URL So please reopen the issue


Question: [Python][Packaging][Release] Missing pyarrow wheels for 17.0.0 release?

Hello @raulcd :wave:
I've set the project limit for `pyarrow` to 40 GB on PyPI. Please be mindful of the frequency of releases at that size.
Have a nice week! :snake:

[Python][Packaging][Release] Missing pyarrow wheels for 17.0.0 release
### Describe the bug, including details regarding any error messages, version, and platform.

For example, release 16.1.0 has 7 Python 3.8 wheels

pyarrow-16.1.0-cp38-cp38-win_amd64.whl
pyarrow-16.1.0-cp38-cp38-manylinux_2_28_x86_64.whl
pyarrow-16.1.0-cp38-cp38-manylinux_2_28_aarch64.whl
pyarrow-16.1.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
pyarrow-16.1.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
pyarrow-16.1.0-cp38-cp38-macosx_11_0_arm64.whl
pyarrow-16.1.0-cp38-cp38-macosx_10_15_x86_64.whl

while 17.0.0 has only 2

pyarrow-17.0.0-cp38-cp38-macosx_11_0_arm64.whl
pyarrow-17.0.0-cp38-cp38-macosx_10_15_x86_64.whl

### Component(s)

Python

Thanks for reporting this. I shared the issue on the mailing list: URL

We reached the quota limit on PyPI and I am waiting for the quota to be increased in order to upload the missing wheels for 3.8, 3.9 and source distribution. The issue on PyPI to track the quota increase is this one: URL

The missing wheels and source distribution have been uploaded to PyPI.

Project Limit Request: pyarrow - 40 GiB
### Project URL

 URL

### Does this project already exist?

- [X] Yes

### New limit

40 GiB

### Update issue title

- [X] I have updated the title.

### Which indexes

PyPI

### About the project

Apache Arrow bindings for Python. Active since 2017.

### How large is each release?

Source: ~1 MiB
Binaries: ~1200 MiB (Windows, macOS, Linux, CPython 3.{8,9,10,11 12})

### How frequently do you make a release?

We do major releases every 3 months and eventually some patch releases.

### Code of Conduct

- [X] I agree to follow the PSF Code of Conduct


Question: Displayed articles are not sorted by date?

@CarlSchwan migrated the admin settings to Vue js already that part is done.
He also took the existing work and cleaned the code a bit, but didn't continue after that probably due to time constraints. You can find a PR with the changes.

@devlinjunker continued the work on the vue-rewrite branch, there are already some commits on that branch, you can find pull requests in his forked repository that are in progress.

That is the latest status.

Displayed articles are not sorted by date
# IMPORTANT
* [x] I have read the CONTRIBUTING.md and followed the provided tips
* [x] I accept that the issue will be closed without comment if I do not check here
* [x] I accept that the issue will be closed without comment if I do not fill out all items in the issue template.

## Explain the Problem
Articles shown in the frontend are not sorted by it's date, but instead of some import date / import ID and grouping.

Please see the video below, you can clearly see that there are articles once "x hours ago", then "days ago", then "hours ago" again, then somewhen "weeks ago"...
I do expect that WHATEVER the source of the news article is, is on top if it was published the most recently.

 URL

This issue seems similar but was closed and stayed closed: < URL

What I quickly tried is changing the `OrderBy` at `nextcloud/apps/news/lib/Db/ItemMapperV2.php` to `pub_date`, then the situation minimally improved, but guess thats the point where changes are needed.

Just for reference, this is how TT-RSS orders newest:

grafik

## Steps to Reproduce
Explain what you did to encounter the issue
1. Open news
2. see the list is not ordered by date and I have to read OLD news first, before seeing other NEW news. Inverse - have to read NEW news, before seeing other OLD news of it!

## System Information
* News app version: 17.0.0
* Nextcloud version: 22.2.3
* Cron type: crontab
* PHP version: 7.4
* Database and version: mariadb 10
* Browser and version: Any browser
* OS and version: Any OS with a browser

----

Would be really cool if you can sort the news by date, instead of some ID made up on your own at database. It would be very fine if the user needs to check "sort by date" / toggle between ID and date, whatever is easiest and works
Thanks!

Seems editing `nextcloud/custom_apps/news/lib/Db/ItemMapperV2.php` does not work anymore, still getting wrong order when viewing a folder:

obrazek

Yea but I think it can be closed, the frontend urgently needs a complete rewrite as the technology it uses is no longer supported and bugs/security issues will stay open. If that is not done the problem will keep existing for forever.
If the frontend gets rebuild this issue will not be relevant anymore.

@Grotax maybe we can add this as todo/sub task for #195? This way it's not 'out of sight'.

Frontend is outdated
The frontend is outdated in various ways and is therefore way harder to maintain then needed:

* Building the js bundle should be done with a few lines of webpack
* After webpack is set up we can move to modularizing the code and use es6 module loading
* Minification should be done with babili since uglifyjs doesnt work with anything remotely ES6 nor newer
* Thirdparty libs should be pulled in entire from npm
* Frontend should probably be migrated to a newer Angular or different technology

For an example webpack file with typescript see URL and URL for integration with karma

---
Want to back this issue? **Post a bounty on it!** We accept bounties via Bountysource.

Some apps are already migrated to Vue. On my personal todo list I have a task trying to rewrite the frontend with Vue.js, but don't know when I'm able to start.
In general I would like to see a new maintainable frontend. :grinning:

@danopz We already have a lot of components ready-to-use for vue. If you want to upgrade the news app, we'll help you with a great joy ;)

I would be glad to help migrating this app to vue.js.

_I migrated the flowupload app to vue (from angular) a month ago (the frontend of this app is obviously more complicated)_

@Grotax @TessyPowder @schaal @SMillerDev @danopz @BernhardPosselt we have a Nextcloud Talk instance also with e.g. a channel for Vue discussions, as well as a "News team" channel – it’s not very active yet but we do have @David-Development @stefan-niedermann and @phedlund already in! :)

And another thing: We are working on a new "Dashboard" at the moment and it would be quite cool if there would be a News app widget for it? :slightly_smiling_face: – see details at URL

I'm not really gonna spend any time on the vue migration, but if you want you can invite me with ` @me.com`

Yea sure but I'm not very active at the moment `info@b-brahmer.de`

How can we get access ? Is it the instance at cloud.nextcloud.com ?

Just invited the 3 of you, you should get an invitation mail. :) Yes it is the instance at cloud.nextcloud.com – and you will also be able to e.g. use the mobile apps for convenience. (You can change your names and also set avatars, but file upload is restricted for guest users.)

_(i changed my username to JonathanTreffler and updated comments mentioning my old username. I wanted to change it for a while and the number of mentions i then need to update only increases)_

The Angular version of news will only be supported until end of 2021.

I updated the admin page to vue here URL , would be great if someone could give it a review

It looks like the frontend is updated to vue as well in v18.2.0

Sorry if I post this here on the wrong place but I was thinking about contributing to the vue rewrite but it's getting a bit messy here with all the branches and PRs. Which one is the most promising/recent one? And can we coordinate work on that?

Are the forks by @CarlSchwan and @devlinjunker different? Is it clear at the moment which to contribute to?

I can't really say. Both are quite in the beginning.

Probably best of you take a look at both and decide for yourself which is the better base to contribute to.

Good evening 👋🏼

I see this has been ongoing for some time, I also see that some of the issues affected by this are still ongoing.

Is there any update on when a rewrite of the frontend will complete?
An issue that was opened about date order was closed as the fix is a rewrite, but thats still in progress.

It'll be done when it is done. You can't plan volunteer time and motivation.

@scottdotau there is URL that has regular contributions. Those contributions have a TODO, so you could pick something up from that.

@SMillerDev appreciate it, obviously didn't read all the replies 🙃.