Question: [Request] Compile version for 32bit - armv7l?

It probably would work fine with pico-arduino but isn't something I can guarantee or test. You'd need to make adjustments to the platform.txt to use the system `arm-eabi-gcc` instead of the packaged one.

[Request] Compile version for 32bit - armv7l
Tool Manager: Installing earlephilhower/toolchain-rp2040-earlephilhower @ 5.100300.220714
INFO Installing earlephilhower/toolchain-rp2040-earlephilhower @ 5.100300.220714
Error: Could not find the package with 'earlephilhower/toolchain-rp2040-earlephilhower @ 5.100300.220714' requirements for your system 'linux_armv7l'

Some users will be wanting to compile pico projects from their raspberry pis, some of which are 32bit only. And even on 64bit devices 32bit Raspbian is the default install by the r-pi foundation - URL

As such it would be great to have a 32bit version of the toolchain.

Related issue: URL

Can you run `gcc -dumpmachine; gcc -v` on your system and paste the output? It should be trivial to add, but I need to know the tuple...

pi@greenpi:~ $ gcc -dumpmachine; gcc -v
arm-linux-gnueabihf
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/arm-linux-gnueabihf/8/lto-wrapper
Target: arm-linux-gnueabihf
Configured with: ../src/configure -v --with-pkgversion='Raspbian 8.3.0-6+rpi1' --with-bugurl=file:///usr/share/doc/gcc-8/README.Bugs --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-8 --program-prefix=arm-linux-gnueabihf- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libitm --disable-libquadmath --disable-libquadmath-support --enable-plugin --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-sjlj-exceptions --with-arch=armv6 --with-fpu=vfp --with-float=hard --disable-werror --enable-checking=release --build=arm-linux-gnueabihf --host=arm-linux-gnueabihf --target=arm-linux-gnueabihf
Thread model: posix
gcc version 8.3.0 (Raspbian 8.3.0-6+rpi1)

This is already there in the Makefile, and the existing binary will work (but P.Io won't install until I re-upload the entire chain.)

See URL

It'll be in the next toolchain build, no actual compilation changes/etc. required...

Great that is good news. When is the next build likely to be?

@earlephilhower sorry to ping you directly, are the packages already uploaded and are just waiting for manual review by the PIO team, or has the push not happened

Running the Arduino toolchain on a RPi 32bit (I assume only RPi3) must really be a test of patience, ouch.

@maxgerhardt it's not been rebuilt yet. Since it's a big D/L for folks I wanted to also get URL and maybe URL in as far as changes before the update.

Worst case I'll do a build by year's end...

ARM64 macOS builds
At the moment it looks like there are no ARM64 (Apple Silicon) builds available from the release page. It would be nice to have these available so I don't need to build them myself.

There's an aarch64 build for 64-bit ARM. I don't know if MacOS supports a Linux compat layer, though, to use it. I'm also pretty sure the Arduino IDE doesn't know what to do w/64b Mac on ARM, either (i.e. even for Win64 is still installs the 32b binaries!).

I do everything via a cross build process and haven't had a Mac since 1996, so if this is important I'll need some assistabnce.

If you look at URL and can add in something that will let Linux x64 build Mac64ARM bits, I'd be happy to include it in the standard release process.

macOS does not support ELF binaries, unfortunately, so those builds won't work.

Sure, I can take a look.

FWIW, I am using PlatformIO and followed the instructions, but it just hung there for a while not doing anything, so I thought something went wrong due to platform architecture. It turns out I was just impatient, and everything took a while to clone and install.

The existing x86_64 builds do work on ARM64 (probably using Rosetta to translate it), so in the meantime that will work. But would be nice to have native builds Eventually™

Hello am looking to build for the pico on Mac. There is support for arm-eabi-gcc on macOS with a package from Homebrew . but it only supports gcc 11. Would GCC 11.3 support be possible? Thanks in advance.

Thanks, @LeSuedois. Unfortunately, I don't think it's an M1 owner I need assistance from here. The issue is that I cross-compile everything, from Raspberry Pi to Win32 to Mac x64 to AARCH64, all in a Docker container on a Linux system. So what needs to be done is figure out how to cross-compile for the M1/MacOS, not actually run builds on one.

It's been a while since I built the container, but IIRC it depends on a manually cross-built XCode. If Apple is still doing XCode, then it's probably a matter of doing the same to the latest bundle.

In the meantime we should try to get Mac x64 version running on M1. Right now it just blows up with
`Could not find the package with 'earlephilhower/toolchain-rp2040-earlephilhower @ ~5.100300.0' requirements for your system 'darwin_arm64'`
Rosetta emulation works pretty well for most apps. Is there a flag or a way to force PlatformIO to just use the X64 version of the toolchain package?

Yup, that'd be the problem right there...it is looking for `darwin_**arm64**` and not falling back to `darwin`. What app are you running?

I'm running PlatformIO with the latest config that looks like

[env:OpenPPG-CRP2040-SP140]
platform = URL
board = sparkfun_promicrorp2040
framework = arduino
board_build.core = earlephilhower

board_build.filesystem_size = 14M ; 14 Mbyte for filesystem and 2 Mbyte for program

I see on the PIO registry that as expected its not showing a version for Mac Arm64
 URL

Ah, I see. Arduino doesn't know what `darwin_amd64` is...it doesn't even know what `winx64` is and that's waaaay older. :(

Is there an override to specify the PIO architecture? the Mac x64 binaries are good and should run (as they do in the IDE) but just a little slower since evidently Rosetta 2 is pretty good stuff.

Alternatively, I might be able to publisy the Mac x64 binaries as Mac ARM64 but that seems like cheating and might cause some grief upstream...

I did some exploration and while I can build most tools for the M1 using the latest `osxcross`, GCC itself isn't yet compatible as a host on the Apple M1 as of now (we need an Apple ARM M1 host, a ARM M0+ target build to make RP2040 binaries). See URL

It looks like only 12.X will support it, and as far as I can see on the GCC changelogs there is no Apple M1 support in as of the 12.2 release. If we move forward to 12.X and the M1 support is merged, I'll revisit. For now, though, unfortunately it's a "hurry up and wait" operation.

FWIW heres what my M1 Pro shows currently

➜ ~ gcc --version
Apple clang version 13.1.6 (clang-1316.0.21.2.5)
Target: arm64-apple-darwin21.4.0

Yup, that's `clang` disingenuously pretending to be `gcc`. I'm sure GCC will be brought to the M1 eventually, it'll just take some time...

The patching is done with in this Ruby file URL but when I try to build this from source I get an error. There are also binaries available which is what is installed by default and what I am using now and they are available here URL

Thanks! I looked at it and it seems like no patching is needed on the 12.2 release of GCC. `darwin20` or `darwin21` looks like it's been added.

Unfortunately that's only 1/2 of what I need. Because I build on Linux I actually need *2* cross-compilers built.
1. Linux x86_64 **host**, AARCH64-darwin20 **target**
2. Using (1), build a AARCH64-darwin20 **host**, Cortex-M0+ **target** compiler.

(1) comes from OSXCross which presently only builds `clang` for the Arm, not `gcc`. I'm not sure that `clang` can be used as the compiler for GCC (it's not used in any of the other cross-builds I do). Will need to see if that's supported.

It turns out there **are** many patches required to make GCC be able to target `aarch-apple-darwin2x`. The Ruby file downloads a clean tarball and then applies patches a **20K++ line patch** from URL

Only GCC 11.3, 12.1, and 12.2 are patched, so the GCC 10 we're using is not supported at all.

I attempted to apply the patch and run an `osxcross` build for GCC 12.2, and it throws massive amounts of compile warnings and fails while build a GCC prerequisite. (On the existing cross builds I don't believe there are more than a handful of warnings thrown for the entire process).

So, that means that even (1) above isn't available,yet.

I'll leave this open but I think this is not going to happen until GH enables their own M1 hosted runners so I can special-case the M1 build...

It looks like there is a arm64 host version of the arm baremetal toolchain available now URL so that might be a better option.

No, I already build `aarch64-linux` just fine. It's the Apple specific one (`aarch64-apple-darwinXX`) which is the problem, sadly.

Maybe this URL
does help, espressif used my cumulative patch to build the native esp32x M1 toolchains for gcc 8.4. Built with URL

I'm getting this error:
Reading symbols from /Users/niccoloperra/Projects/tesla-hud/.pio/build/pico/firmware.elf...
PlatformIO Unified Debugger -> URL
PlatformIO: debug_tool = cmsis-dap
PlatformIO: Initializing remote target...
dyld[11467]: Library not loaded: /usr/local/opt/libusb/lib/libusb-1.0.0.dylib
 Referenced from: /Users/niccoloperra/.platformio/packages/tool-openocd-rp2040-earlephilhower/bin/openocd
 Reason: tried: '/usr/local/opt/libusb/lib/libusb-1.0.0.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/libusb/lib/libusb-1.0.0.dylib' (no such file), '/usr/local/opt/libusb/lib/libusb-1.0.0.dylib' (no such file), '/usr/local/lib/libusb-1.0.0.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/usr/lib/libusb-1.0.0.dylib' (no such file, not in dyld cache)

 Did you have any similar issues?

You need the Intel format USB library and only have the ARM64 version.

Back in the day, when going from 68k->PPC and IIRC PPC->x86, the Mac had "fat binaries" with code for both architectures in it, so a library or app could run on either architecture. Does it have it for the x86_64->arm64 transition? Can you install x86_64 versions of libraries in some other directorylike /usr/local/opt/x86_64 or something (like Linux uses for multiarch)?


Question: E523 when using clever-f?

In `:h map- `

Be very careful about side effects! The expression is evaluated while
obtaining characters, you may very well make the command dysfunctional.
Therefore the following is blocked for mappings:
- Changing the buffer text |textlock|.
- Editing another buffer.
- The |:normal| command.
- Moving the cursor is allowed, but it is restored afterwards.
- If the cmdline is changed, the old text and cursor position are restored.
If you want the mapping to do any of these let the returned characters do
that. (Or use a | | mapping instead.)

E523 when using clever-f
Hello,
I am using clever-f and I got this issue

<img width="1715" alt="image" src=" URL

It happens during the initialization of LSP server and after the initialization, the issue is gone. To reproduce, open any `lua` file (using `lua` because `lua` server needs longer time to initialize, so it will be easier to reproduce), then before the finish of the LSP server initialization, press `f` plus some char, then this issue appears.

Thank you very much.

@rockyzhang24 it looks like your figet.nvim is outdated, because L216 does not read like `vim.schedule`. Are you sure it is on the latest version?

Thanks.
No, it is the latest fidget. I update my plugins everyday. My neovim is the latest nightly (NVIM v0.7.0-dev+1439-gda31e953b)

Btw, the issue info is not the same for different filetype.

In a `go` file, I got
<img width="1715" alt="image" src=" URL

Sorry, I misread the stacktrace. You are right -- I can also reproduce the bug.

Indeed `vim.schedule` is from L386 (the bottommost line) and L216 reads

api.nvim_win_set_option(self.winid, "winblend", options.window.blend)

The problem is that the "deferred" `fidget:fmt()` call can still be executed by the event loop during the execution of `getchar()` (see URL I am looking into why.

Thank you so much. I am not sure whether this issue is caused by `clever-f`. I already opened an issue there ( URL Also, I am not sure whether this issue is related to URL i.e., caused by ` ` and `getchar()`. I have no ideas.

Also worth checking out a comment by @kevinhwang91 ( URL -- I use a similar plugin that maps `f`, namely `quick-scope`. Or @kevinhwang91's `nvim-fFHighlight` might be another alternative.

Clever-f is not actively maintained these days, so you might want to switch to another similar plugin as a workaround. It's a bug that clever-f needs to fix, or neovim might be responsible for scheduling as `vim.schedule` is supposed to defer the execution *when it is available*.

Okay. Thank you for your suggestions.
I know this new plugin `nvim-fFHighlight` but it not supports `tT` highlight.
For `quick-scope`, I tried it shortly before and I am worried its performance because it analyze each word in a line, not just the char I want to jump to.

@wookayin Thank you so much for your investigation. Both fidget and clever-f are my must-have. Hope this can be fixed soon. Thanks again.

Yes, it is perfectly fine. Because E523 errors happens due to a 'textlock' (see `:h textlock`) and it only pertains to drawing of the fidget windows. Any internal states and data will remain intact. As you could test out, the fidget windows will simply won't be updated nor be spinning while it is in the operator-pending mode.

@wookayin Okay, thank you for explaining it in detail. I just tested it, and I found another glitch.

To reproduce it:
1. Open a `lua` file (because `lua` LSP takes longer to initialize)
2. Press `d` followed by `f`, now we are entering `Operator Pending` mode
3. Wait here without giving a char to `f`. We can see that fidget progress update and spinning pause as expect. Keep waiting, and we can see fidget left some ghost chars even though it has finished in background.4.

 ` immediately after the ghost chars appear. The ghost will still be there and won't be removed any more. However, if we press ` ` after a while, the error shows again

<img width="1721" alt="image" src=" URL

Thank you!

@rockyzhang24 Good catch. Can you try again in the updated PR?

Because `nvim_get_mode` is only for RPC API, `blocking = true` is always for Lua.

I don't think it's appropriate for fidget to fix this bug. There're maybe a lot of other async functions to encounter this issue. From the upstream is too hard to fix. From the clever-f avoid using must be refactored the cost is too expensive.

Again, the problem is that a function deferred by `vim.schedule(..)` will still be executed while operator is pending. We could safely skipping rendering, but when it comes to kill or closing, this workaround doesn't seem to work because it will lead to resource leakage or weird race condition. Fidget cannot easily tell when it is really available, despite the use of `vim.schedule`.

I think this requires an upstream fix, so no good solution for now. For the time being, I think #69 can be still merged to avoid most of E523 errors, and we can later improve the remaining case when fidget is trying to close.

@j-hui Any thoughts?

@rockyzhang24 I rewrote the logic and we can ignore the error when fidgets would be closed as well. This will make the annoying error not happen any more. However it cannot 100% eliminate "ghost chars" as I explained earlier; but we have some workaround like `:FidgetClose` so this should be usable enough.

@wookayin Thank you so much for digging deeper into this issue and trying to come up with solutions. It looks like we have to wait for the upstream fix to make this issue solved completely. Could you please open an issue upstream? If I do, I'm afraid I cannot describe it clearly.

`nvim_buf_set_lines` reported E523 in map-expr
### Neovim version (nvim -v)

NVIM v0.5.0

### Vim (not Nvim) behaves the same?

no

### Operating system/version

macOS 11.0.1

### Terminal name/version

kitty 0.18.1

### $TERM environment variable

screen-256color

### Installation

homebrew

### How to reproduce the issue

1. open neovim `nvim`
2. define a function that uses

 :function X()
 : let buf = nvim_create_buf(v:false, v:true)
 : call nvim_buf_set_lines(buf, 0, -1, v:true, ["test", "text"])
 : let opts = {'relative': 'cursor', 'width': 10, 'height': 2, 'col': 0, 'row': 1, 'anchor': 'NW', 'style': 'minimal'}
 : let win = nvim_open_win(buf, 0, opts)
 :endfunction

3. define an `expr` mapping by `:map x X()`
4. enter normal mode and press `x` key.

### Expected behavior

No error occurs

### Actual behavior

Error occurs:

Error detected while processing function X:
line 2:
E5555: API call: E523: Not allowed here
Press ENTER or type command to continue

Note that with "normal" bindings like `:map t :call X() ` actually works.

@zeertzjq Thanks for clarification.

I was actually trying to port an input method plugin ywvim's calling to vim's popup window to floating window. The plugin would use popup window to display the choices around the cursor, and popup window's APIs are called(and legitimate) in there `map-expr` functions. e.g.

image

Is there any workaround in neovim for implementing this?


Question: Tooltip for TV bottom left not appearing anymore at some time?

I might have remembered wrong about old TVs... But for me they are simply 5 "switches". Press one and the others are deactivated.
So power off will switch off other buttons too etc.

Of course this can be changed. Guess that code is around 15 yrs old now ;-)

Tooltip for TV bottom left not appearing anymore at some time
When playtesting with the version built by Ronny here it happened at some time that the tooltip for the TV on the bottom left didn't appear anymore.

Screenshot from 2023-03-09 18-57-55

It happened only once, and I guess it's very rare. But I don't know how to reproduce. I made a savegame but it wasn't reproducing that effect, the tooltip worked again after restart.

If this happens again: can you check if you cannot open up the ESC-Menu in that case?

the ESC-Menu thing is something rarely happening too - I even added a debug-logging for it but was not motivated enough to track it down (means trying often enough to make it fail :D).

I have never encountered this problem. I assume the other tooltips in the lower region (audience, betty...) do not stop working at this point? Basically, they are all treated identically wrt. update and rendering...
Even if you turn off the TV, you don't get the tooltip for "turning it on again"?

It happened again.

ESC menu is still working as usual.

The other tooltips in the lower center info region are working.

*That* led me to another bug: URL

The error is caused by very long programme titles, e.g. "Dynamit - Das Magazin Spezial: Prince Carl und Lady Dee (5/5): Dynamit Prince Carl und Lady Dee Spezial - Schlussfolge (Block 1/1)".

If the tooltip width exceeds the screen width, the tooltip is moved to the left (base.framework.tooltip line 154). There is no guard that the resulting x position remains positive. And possibly the calculation of the delta is also buggy, because with x-position 41, tooltip width 830 and screen width 800 the resulting tooltip x-position after moving is -3839.

Potential TODOs:
* check and possibly fix calculation of tooltip-deltas
* improve framework, so that original x-position is stored and is restored if title/content change (moving tooltip is only temporary)
* tooltip title is single line, so programme tooltip should possibly ensure the title passed has a maximum length

While this of course is a bug in the framework I have to ask:

Seriously? I mean ... seriously? Titles should be way way shorter - so that the ellipsis (...) does not cut away important parts and the title is still distinguishable from other elements of that series/franchise.

I guess the length of the title is due to it's being a series episode. I would not extend the framework to allow multi-line tooltip headers! This is an absolute border case. Ensuring that the tooltip does not permanently disappear from the screen should be enough. (Making sure the resulting x coordinate is not smaller than 0 and restoring it to its original value should be enough).

Yeah sure that is redundant in that case here and can be shortened.

But in general, such long titles (especially in series when combining general and episode title) can exist.

Lower left TV breaks when switching off and on again
"Have you tried switching it off and on again?"

Sure, but that's exactly when the TV breaks indefinitely.

The TV just remains off, and can't be switched on again.
*However*, when hovering over the channel buttons, the TV picture for that program will still be shown. Which may be an additional bug, I'm not sure.

Even after saving and reloading a game, the situation is still the same.

Should be easily reproducible, seems the functionality is broken in general. If not, I'll provide the savegame.

Just astonishing that no-one reported this feature. Who ever needs to switch off the TV after all? I wondered about that "feature" that you can switch off the TV at all already back then in Mad TV. What's the purpose? Less distraction so you can "focus" better? `:)`

So far, I cannot reproduce the problem (Linux). For a newly started game turning the TV on/off does not break the hover (note that switching on is not done using the off-button, but using a channel button).

My assumption is that even a savegame might not help. I don't know if I can make myself clear. The savegame does not contain absolutely all data and not all game status is persisted and taken from that data. That is, if something is broken internally and not "repaired" by loading the game, it remains broken (e.g. maybe the broken hover for the TV). Likewise, not everything broken is transferred to the savegame. So maybe the hover status is not persisted and loading in a freshly started game will not have the problem anymore.

The channel tooltip is only visible when the game is not "paused".

If the game is not paused the game logic for "button reaction" is a simple check of the "mouse position" (for hover) and "click" (for activation)

Maybe that "paused" thing is what borks it for @scr0llbaer9000

But not when clicking the power button, right?

Is there a misunderstanding here?

The expectation is absolutely that, just as in real life, when you switch the TV off with the main black power button, you can also switch it on again with the black button (as was the case in Mad TV afair). And I figure it is still reproducible that this doesn't work?

I indeed was wrong in OP that the TV was broken "indefinitely", as it can be switched on again by clicking on one of the channel buttons. For some reason it didn't dawn on me to try that. `:)`

yeah the 4 channel buttons can be switches, but the power button is its own thing.

When switching on with the power button, the last "active" channel can be shown again.
(In real life, channel buttons on most TVs weren't switches either though, the last "state" was not remembered, and channel 1 was shown when switching on.)

For simplicity, my suggestion would be to show the channel of the "active" player when switching on, again.


Question: I have a lot of things I want you to improve?

@tahayparker you need to start the app at least once, optionally there is a setting hidden under the "advanced" toggle where you can enable context menu.
Also, LocalSend is available under "Send to" not as an share option in Windows 10/11

I have a lot of things I want you to improve
### Is your feature request related to a problem? Please describe.

1.[pc] The minimized icon in the lower right corner should be changed to: left click to open directly, right click to pop up options

2.[pc] Add shortcut send in the right-click menu

3.[pc, android] When there is one device connected, it will be sent directly without confirming the device. When the device is larger than one or there is no device, a pop-up window will appear to select the device

4.[pc, android] After sending "A" file and then sending "B" file, you need to delete the previously added a, I think you need a switch that does not keep the history added

5.[pc, android] show real-time transfer speed

6.[android] Android client save location can be set

### Describe the solution you'd like

The software looks good overall, but many functions are still unsatisfactory

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

5. Real time speed is always shown as soon as 5 MB has been sent, the speed is too volatile for small transfers.

1. Added in `v1.8.0`
2. Duplicate of #115
3. I don't know if this is confusing / unexpected behaviour
4. Added in `v1.8.0`
5. See previous comment
6. This is technically difficult because of the new scoped storage permissions.

Windows context menu (right click menu)
Hi, I love the application. This is a feature request.

Can you add this program into the Windows 10/11 context menu so you can quickly right click a file and share?

yes pls add this option then finally i can switch from send anywhere .. btw love ur work i tried many but ur app was the one i ws looking for :) keep it up bro

Flutter currently doesn't offer an easy way to add context menus.

This could possibly be implemented by adding a shortcut to the app in the SendTo folder, and reading the path to the file/folder from the command line arguments (in the `main` function).

I came to open a issue specifically asking for this implementation. Big support

any update on this? this issue has been stale for 4 months now

Hi Tien, when can we expect this feature to be part of a release?

Thank you very much Tien
I would like to express my gratitude, and I recognize the effort that goes into implementing such a feature.

能否支持Win11新版右键菜单的系统分享？
Can the system sharing of the new right-click menu of Win11 be supported?
image

Hi @Tienisto
I believe that this issue was fixed and is part of v1.15.0
However, I do not see the context menu entry

i see, i had disabled the "send to" as i didn't need it
would it be possible to create the entry in the "main" context menu rather than in the "send to" submenu?

localsend_2024-08-11_09-28-25
i enable it, but Shortcuts in the "Send To" folder not appearing in the menu.

You can try manually modifying the registry to add LocalSend's sending function to the main context menu of the right-click.

I have tried to modify the registration expression to put the send directive in the context menu, you can try to create a txt document type the following (if you localsend is the default installation address) and change the txt suffix to reg.
:Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\*\shell\LocalSend]
@="Send with LocalSend"
"Icon"="C:\\Program Files\\LocalSend\\localsend_app.exe"

[HKEY_CLASSES_ROOT\*\shell\LocalSend\command]
@="\"C:\\Program Files\\LocalSend\\localsend_app.exe\" \"%1\""

[HKEY_CLASSES_ROOT\Directory\shell\LocalSend]
@="Send with LocalSend"
"Icon"="C:\\Program Files\\LocalSend\\localsend_app.exe"

[HKEY_CLASSES_ROOT\Directory\shell\LocalSend\command]
@="\"C:\\Program Files\\LocalSend\\localsend_app.exe\" \"%V\""

[HKEY_CLASSES_ROOT\Directory\Background\shell\LocalSend]
@="Send with LocalSend"
"Icon"="C:\\Program Files\\LocalSend\\localsend_app.exe"

[HKEY_CLASSES_ROOT\Directory\Background\shell\LocalSend\command]
@="\"C:\\Program Files\\LocalSend\\localsend_app.exe\" \"%V\""


Question: Zero injection?

@kiss81 It should be possible if we knew the zigbee commands for it. Sadly we don't. Someone with an official ECU can provide this data with a zigbee sniffer, so far no volunteer has come forward.

Zero injection
I'm specifically interested in whether it's feasible to control the microinverter's zero injection mode via the ESP ECU. Thanks.

If you know the zigbee command it would be doable via the console. Sadly i don't know the command.

How can I acquire the command? What was your method for decoding the Zigbee data? Thank you very much.

This question has been asked before.
read here and
to get an idea how the reversed-engineering went, read here

In short: Someone with an official ECU needs to sniff the communication ecu inverter.

feature request: turn off / turn on (export limit)
Not sure if this is possible, but some inverters support a temporary export limit. (turn off the inverter)
Is there any chance this could work? In the Netherlands we sometimes have negative prices an hour a day so you have to pay for producing electricity.
I am using this on a qs1 by the way.

I asked on a forum that's working on integration of the official ecu. I do hope one has a sniffer...


Question: JDBC database shutdown on disconnect.

Thanks for raising this! There were a number of changes around automatically closing JDBC connections and caching them in the 0.6.0 release, so it's possible something broke

JDBC database shutdown on disconnect.
A duckdb connector has been implemented recently on DBeaver. However, the database does not shutdown on disconnect.
Is there a JDBC driver feature similar to the R package where we can say `shutdown = TRUE`?

I also ran into this bug and couldn't open my duckdb from Python, not even with `read_only=True`.

Disconnect in DBeaver didn't help, only closing the DBeaver application completely resolved the issue.

Is there anyone that can fix the DBeaver connector to disconnect properly?

In [82]: con = duckdb.connect("my-db.duckdb", read_only=True)
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
Input In [82], in ()
----> 1 con = duckdb.connect("my-db.duckdb", read_only=True)

RuntimeError: IO Error: Could not set lock on file "my-db.duckdb": Resource temporarily unavailable

@cdeil are you able to test if this still occurs with the latest master build? We now shutdown the database instance when the last connection is closed

I'm not sure how easy it would be for me to reproduce the issue and then build and try the master.
If you or others in the dev team can't easily do that, I could try, let me know.
Or alternatively - when is your next stable release? Could also wait for that and try again.

@cdeil are you able to confirm if this behaves as expected for you with 0.6.0?

I updated the Python version as well as the driver in DBeaver to duckdb 0.6.0.

I can confirm that after closing the DB connection in DBeaver, I can now open the file from Python. So that bug is not there any more.

---

However I'm still seeing that after opening a DuckDB in DBeaver, I still can't open it in Python in read-only mode:

In [2]: duckdb.connect("/Users/cdeil/code/hc/hpred/data/cache/sensor.duckdb", read_only=True)
---------------------------------------------------------------------------
IOException Traceback (most recent call last)
Input In [2], in ()
----> 1 duckdb.connect("/Users/cdeil/code/hc/hpred/data/cache/sensor.duckdb", read_only=True)

IOException: IO Error: Could not set lock on file "/Users/cdeil/code/hc/hpred/data/cache/sensor.duckdb": Resource temporarily unavailable

---

And now I noticed a purely DBeaver-related bug (without Python) which can be reproduced like this:

1. Open a DuckDB (works).
2. Disconnect (works).
3. Try to connect again - fails with error:

Connection Error: Instance with path: /Users/cdeil/code/hc/hpred/data/cache/sensor.duckdb already exists.

So it's still the case that I completely have to close the DBeaver app to continue working. Somehow the JDBC driver is left in a bad state after disconnecting that prevents it to re-connect later.

Are you opening the database in read only mode in DBeaver when you open it in Python (also in read only)?

We've had a couple of reports of that DBeaver issue yes, there were a number of changes that might have caused it so I'll need to do some investigation

I was using normal read/write mode in DBeaver and read-only mode in Python.

Unfortunately you can only have one client connected to a database if you're using a write connection. Multiple connections is only (presently) supported in read only mode

Looks like your other issue is a duplicate of #5371

OK then I think this issue should be closed.

@travis-leith as original reporter - do you agree?

---

Is it a known or valid feature request for DuckDB to allow other readers even if a writer has a connection open? Or will it never be possible with how DuckDB works?

My use case is simply that I use DBeaver as a DB GUI to check content, while I have a Python script writing to it, that's when I ran into these issues.

I have not had the pleasure of using duckdb for a while now. But going by the above discussion, looks like this can be closed, and possibly a feature request for read only connections in the presence of a read/write connection.

Instance caching prevents multiple connections, closing and re-opening with JDBC
### What happens?

It looks like the instance caching is failing to allow either multiple connections, or closing the connection and subsequently opening the database again.

### To Reproduce

This throws an error:

import java.sql.Connection;
import java.sql.DriverManager;

class duckdb {
 public static void main(String[] args) {
 try {
 Connection conn = DriverManager.getConnection("jdbc:duckdb:test-temp.db");
 Connection conn2 = DriverManager.getConnection("jdbc:duckdb:test-temp.db");

 conn.close();
 conn2.close();
 } catch (Exception e) {
 System.out.println(e);
 e.printStackTrace();
 }
 }
}

The error thrown:

java.sql.SQLException: Connection Error: Instance with path: /Users/pellsworth/test-duckdb/test-temp.db already exists.
 at org.duckdb.DuckDBNative.duckdb_jdbc_startup(Native Method)
 at org.duckdb.DuckDBDatabase. (DuckDBDatabase.java:28)
 at org.duckdb.DuckDBDriver.connect(DuckDBDriver.java:36)
 at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
 at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:251)
 at duckdb.main(duckdb.java:8)
 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.base/java.lang.reflect.Method.invoke(Method.java:566)
 at jdk.compiler/com.sun.tools.javac.launcher.Main.execute(Main.java:404)
 at jdk.compiler/com.sun.tools.javac.launcher.Main.run(Main.java:179)
 at jdk.compiler/com.sun.tools.javac.launcher.Main.main(Main.java:119)

(it also will throw the same error if you try to close the first connection before opening the second).

### OS:

OSX, x64

### DuckDB Version:

0.6.0

### DuckDB Client:

Java

### Full Name:

Paul Ellsworth

### Affiliation:

Tenable

### Have you tried this on the latest `master` branch?

- [X] I agree

### Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

- [X] I agree

(I might also just be totally missing how it's supposed to work :) )

Looks like this is a bug introduced in URL maybe @Mytherin can take a look?


Question: nixos/glance: glance-start-pre: install: invalid user 'glance'?

@pinpox The `-q` was already there, and I'm just removing the `-s`. Sorry I didn't make that clearer.

With the extra configuration:

ini
$ systemctl cat NetworkManager-wait-online.service
# /etc/systemd/system/NetworkManager-wait-online.service
[Unit]
Description=Network Manager Wait Online
Documentation=man:nm-online(1)
Requires=NetworkManager.service
After=NetworkManager.service
Before=network-online.target

[Service]
# `nm-online -s` waits until the point when NetworkManager logs
# "startup complete". That is when startup actions are settled and
# devices and profiles reached a conclusive activated or deactivated
# state. It depends on which profiles are configured to autoconnect and
# also depends on profile settings like ipv4.may-fail/ipv6.may-fail,
# which affect when a profile is considered fully activated.
# Check NetworkManager logs to find out why wait-online takes a certain
# time.

Type=oneshot
ExecStart=/nix/store/r7fwxkxbgmwyrff5n03r9nchzskj59n9-networkmanager-1.40.16/bin/nm-online -s -q
RemainAfterExit=yes

# Set $NM_ONLINE_TIMEOUT variable for timeout in seconds.
# Edit with `systemctl edit NetworkManager-wait-online`.
#
# Note, this timeout should commonly not be reached. If your boot
# gets delayed too long, then the solution is usually not to decrease
# the timeout, but to fix your setup so that the connected state
# gets reached earlier.
Environment=NM_ONLINE_TIMEOUT=60

[Install]
WantedBy=network-online.target

# /nix/store/2ma3yvlqmj94dr6v24nkwmspgxfpfnmq-system-units/NetworkManager-wait-online.service.d/overrides.conf
[Unit]

[Service]
Environment="LOCALE_ARCHIVE=/nix/store/g135laybfk1c7bm2zhrh7x6dv884qxal-glibc-locales-2.35-224/lib/locale/locale-archive"
Environment="PATH=/nix/store/l6jgwxkc3jhr029vfzfwzcy28iyckwsj-coreutils-9.1/bin:/nix/store/gn1s1s5z19cf0wiir2cd38jckcjc6kn6-findutils-4.9.0/bin:/nix/store/pvb117r7fhwb08717ks21a6y>
Environment="TZDIR=/nix/store/z0kg1c0f8fx6r4rgg5bdy01lb2b9izqg-tzdata-2023a/share/zoneinfo"

ExecStart=
ExecStart=/nix/store/r7fwxkxbgmwyrff5n03r9nchzskj59n9-networkmanager-1.40.16/bin/nm-online -q

nixos/glance: glance-start-pre: install: invalid user 'glance'
### Nixpkgs version

- Unstable (25.05)

### Describe the bug

After URL was merged glance module does not start and throws an error message. (see relevant log output)

My configuration does not use any secrets.

It seems that when this PR was implemented it was not tested with a configuration that uses no secrets at all and therefore does not create a `glance` user owned secrets with sops-nix or other methods.

### Steps to reproduce

Use glance nixos module with no secrets while having no `glance` user on your system

### Expected behaviour

The glance module works with no secrets defined and no 'glance' user created using the means outside of the `nixos/glance` module.

### Screenshots

_No response_

### Relevant log output

console
kgl3a8vqw0ly5k0ff86y0k1kz3aw0g61-glance-start-pre[261578]: install: invalid user 'glance'

### Additional context

_No response_

### System metadata

 - system: `"x86_64-linux"`
 - host os: `Linux 6.12.29, NixOS, 25.11 (Xantusia), 25.11.20250520.2795c50`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.28.3`
 - nixpkgs: `/nix/store/p893dkrzm5rxvhnqh092prgi1a7dzmcy-source`

### Notify maintainers

@drupol @gepbird

---

**Note for maintainers:** Please tag this issue in your pull request description. (i.e. `Resolves #ISSUE`.)

### I assert that this issue is relevant for Nixpkgs

- [x] I assert that this is a bug and not a support request.
- [x] I assert that this is not a duplicate of an existing issue.
- [x] I assert that I have read the NixOS Code of Conduct and agree to abide by it.

### Is this issue important to you?

Add a :+1: [reaction] to [issues you find important].

[reaction]: URL
[issues you find important]: URL

It uses `$USER` which is provided by systemd using DynamicUser IIUC, I expect it to work without secrets too

The NixOS test, which had a machine with only `services.glance.enable = true` config also passes

Can you the config for a minimal repro please? I'll try to look into this more deeply a few days later, I'm a bit busy right now

I tried to reproduce it in a VM but couldn't, this is what I tried:
- add some glance config from the example

 glance config

nix
 services.glance = {
 enable = true;
 settings = {
 pages = [
 {
 columns = [
 {
 size = "full";
 widgets = [
 {
 type = "calendar";
 }
 ];
 }
 ];
 name = "Calendar";
 }
 ];
 };
 };

- build with nixpkgs at b4667dba16e62bd6f589383d1d23c4084545ad6c (before the glance secret PR), switch to the config
- observe new glance service starting without issues
- build with nixpkgs at aaf817bb5b0d2c7bb7892fb158292805f8927d65 (after the glance secret PR), switch to the config
- observe that glance-start-pre and replace-secret packages were added (via nvd) as expected, but the glance systemd service is still running without errors

It's also possible that not URL is causing the issue but something else (for example systemd changes, I haven't done any search for these), it would be helpful if you could share the old and new nixpkgs revision that caused this issue @jaredmontoya @wyndon. The best would be to link the commit to your config where you found this bug.

github:nixos/nixpkgs/nixos-unstable

Before: d89fc19e405cb2d55ce7cc114356846a0ee5e956
After: 2795c506fe8fb7b03c36ccb51f75b6df0ab2553f

Thanks, but unfortunately I still couldn't reproduce it :(

When I switched from your before revision to your after, I saw glance getting updated (0.8.1 -> 0.7.13), but nothing else relevant really and the service restarted successfully. Also on your before revision, the glance secret PR is already included, so it wasn't introduced between the 2 commits

Maybe it really is caused by systemd changes or something else like that.

Good news, I could finally reproduce it. However it's inconsistent, it only failed 3 out of 10 times. Switching to a system built with a bad commit (2c8d3f48d33929642c1c12cd243df4cc7d2ce434, 2025-04-02, I picked a way older one than y'all reported just to be sure) then a good commit (063f43f2dbdef86376cc29ad646c45c46e93234c, 2025-05-23).

When trying to reproduce this between the suspected PR and a commit right before it, the issue wasn't there 20 out of 20 times. It's possible that it depends on the timing: when I can repro it sometimes, the activation script runs for a few seconds as months of changes require many services to be restarted, but switching between PR and right before it is very fast and the issue doesn't have enough time to manifest

I hope this is not another URL

cc @NixOS/systemd, is there wrong with URL Sometimes when updating nixpkgs to a newer version we get `kgl3a8vqw0ly5k0ff86y0k1kz3aw0g61-glance-start-pre[261578]: install: invalid user 'glance'` for the first time, after that restarting the service works fine forever™ (unless we downgrade to an older nixpkgs and upgrade again)

I'm going to bet on some kind ofrace condition with `nsncd` . As that's the way we provide NSS resolution in NixOS. But it's a bit odd as it's socket activated.

Anything in the logs about nsncd when you reproduce it? Maybe switch-to-configuration also restarting it?

Yes, `nscd` service is restarted when switching from that months old nixpkgs to an up-to-date one, here are the logs with precise time stamps:

 log of glance.service

console
May 25 10:11:27.441753 gepvm systemd[1]: Stopping Glance feed dashboard server...
░░ Subject: A stop job for unit glance.service has begun execution
░░ Defined-By: systemd
░░ Support: URL
░░
░░ A stop job for unit glance.service has begun execution.
░░
░░ The job identifier is 8443.
May 25 10:11:27.449282 gepvm systemd[1]: glance.service: Deactivated successfully.
░░ Subject: Unit succeeded
░░ Defined-By: systemd
░░ Support: URL
░░
░░ The unit glance.service has successfully entered the 'dead' state.
May 25 10:11:27.450093 gepvm systemd[1]: Stopped Glance feed dashboard server.
░░ Subject: A stop job for unit glance.service has finished
░░ Defined-By: systemd
░░ Support: URL
░░
░░ A stop job for unit glance.service has finished.
░░
░░ The job identifier is 8443 and the job result is done.
May 25 10:11:31.498145 gepvm systemd[1]: Starting Glance feed dashboard server...
░░ Subject: A start job for unit glance.service has begun execution
░░ Defined-By: systemd
░░ Support: URL
░░
░░ A start job for unit glance.service has begun execution.
░░
░░ The job identifier is 9158.
May 25 10:11:31.541156 gepvm rv53hcsa0v4r61wi4wvgc9qf4asj9165-glance-start-pre[7906]: install: invalid user 'glance'
May 25 10:11:31.543474 gepvm systemd[1]: glance.service: Control process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: URL
░░
░░ An ExecStartPre= process belonging to unit glance.service has exited.
░░
░░ The process' exit code is 'exited' and its exit status is 1.
May 25 10:11:31.543676 gepvm systemd[1]: glance.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: URL
░░
░░ The unit glance.service has entered the 'failed' state with result 'exit-code'.
May 25 10:11:31.545265 gepvm systemd[1]: Failed to start Glance feed dashboard server.
░░ Subject: A start job for unit glance.service has failed
░░ Defined-By: systemd
░░ Support: URL
░░
░░ A start job for unit glance.service has finished with a failure.
░░
░░ The job identifier is 9158 and the job result is failed.

 log of nscd.service

console
May 25 10:11:27.428515 gepvm systemd[1]: Stopping Name Service Cache Daemon (nsncd)...
░░ Subject: A stop job for unit nscd.service has begun execution
░░ Defined-By: systemd
░░ Support: URL
░░
░░ A stop job for unit nscd.service has begun execution.
░░
░░ The job identifier is 8436.
May 25 10:11:27.430064 gepvm systemd[1]: nscd.service: Deactivated successfully.
░░ Subject: Unit succeeded
░░ Defined-By: systemd
░░ Support: URL
░░
░░ The unit nscd.service has successfully entered the 'dead' state.
May 25 10:11:27.431271 gepvm systemd[1]: Stopped Name Service Cache Daemon (nsncd).
░░ Subject: A stop job for unit nscd.service has finished
░░ Defined-By: systemd
░░ Support: URL
░░
░░ A stop job for unit nscd.service has finished.
░░
░░ The job identifier is 8436 and the job result is done.
May 25 10:11:31.515812 gepvm systemd[1]: Starting Name Service Cache Daemon (nsncd)...
░░ Subject: A start job for unit nscd.service has begun execution
░░ Defined-By: systemd
░░ Support: URL
░░
░░ A start job for unit nscd.service has begun execution.
░░
░░ The job identifier is 8955.
May 25 10:11:31.545377 gepvm nsncd[7889]: May 25 08:11:31.545 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/sock>
May 25 10:11:31.548892 gepvm systemd[1]: Started Name Service Cache Daemon (nsncd).
░░ Subject: A start job for unit nscd.service has finished successfully
░░ Defined-By: systemd
░░ Support: URL
░░
░░ A start job for unit nscd.service has finished successfully.
░░
░░ The job identifier is 8955.

As expected, nothing changed since then with how ns(n)cd get started.

Oh, ok.
I thought atomic updates were part of the `nixos-rebuild switch` functionality.
But I guess it's all `deploy-rs` magic rollback's doing in my case.

Somehow I didn't know about it (even though it's widely used in docker, systemd and nixpkgs), I opened a PR, thanks for your idea!

I think we should keep this issue open for tracking the purposes of the root cause.

NetworkManager's nm-online kills nixos-rebuild
### Describe the bug

The systemd service `NetworkManager-wait-online.service` can prevent `nixos-rebuild` from succeeding:

warning: the following units failed: NetworkManager-wait-online.service

× NetworkManager-wait-online.service - Network Manager Wait Online
 Loaded: loaded (/etc/systemd/system/NetworkManager-wait-online.service; enabled; vendor preset: enabled)
 Drop-In: /nix/store/k5yq51spcggip2h6aq1y0bydkpr4zahc-system-units/NetworkManager-wait-online.service.d
 └─overrides.conf
 Active: failed (Result: exit-code) since Tue 2022-07-05 10:18:52 NZST; 36ms ago
 Docs: man:nm-online(1)
 Process: 1258376 ExecStart=/nix/store/b4yhg54s70i0v0k1qnnv8vnja6018yrh-networkmanager-1.38.2/bin/nm-online -s -q (code=exited, status=1/FAILURE)
 Main PID: 1258376 (code=exited, status=1/FAILURE)
 IP: 0B in, 0B out
 CPU: 22ms

Jul 05 10:17:52 nixos systemd[1]: Starting Network Manager Wait Online...
Jul 05 10:18:52 nixos systemd[1]: NetworkManager-wait-online.service: Main process exited, code=exited, status=1/FAILURE
Jul 05 10:18:52 nixos systemd[1]: NetworkManager-wait-online.service: Failed with result 'exit-code'.

This service runs `nm-online -s -q`, and the `nm-online` man page says:

-s | --wait-for-startup
 Wait for NetworkManager startup to complete, rather than waiting for network connectivity specifically. Startup is
 considered complete once NetworkManager has activated (or attempted to activate) every auto-activate connection
 which is available given the current network state. This corresponds to the moment when NetworkManager logs "startup
 complete". This mode is generally only useful at boot time. After startup has completed, nm-online -s will just
 return immediately, regardless of the current network state.

 There are various ways to affect when startup complete is reached. For details see NetworkManager-wait-
 online.service(8).

I am not familiar with this tool, but my experience is that after my laptop has been up for some time (e.g. days), `nm-online` will often return an error code rather than correctly determine the network is up, thus killing any future `nixos-rebuild` commands.

### Steps To Reproduce
Steps to reproduce the behavior:

1. Get your machine in a state where `nm-online -s -q` does not return success (not sure how to do this on demand).
2. Witness `nixos-rebuild` failure.

### Expected behavior

`nixos-rebuild` should not fail due to an erroneous network check.

### Additional context

This is tricky as it is not a `nix` issue per se but rather an issue with a presumably flaky systemd service. It is easy enough to disable this service manually:

nix
systemd.services.NetworkManager-wait-online.enable = false;

And perhaps this is the best solution. But a number of my coworkers all ran into this issue independently, so I thought it merited an issue for discoverability, if nothing else. My gut reaction is that a flaky check should probably not be required by default, but I don't know enough about this service's importance/fragility to say.

This issue was noticed only recently, both on `nixos-unstable` and `nixos-22.05`.

### Notify maintainers

### Metadata
Please run `nix-shell -p nix-info --run "nix-info -m"` and paste the result.

console
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.47, NixOS, 22.05 (Quokka)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.8.1`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Thanks!

This issue has been mentioned on **NixOS Discourse**. There might be relevant details there:

 URL

This is related to udev not initializing devices. NetworkManager never completes startup because a WireGuard interface is never initialized by udev. A workaround is just putting the affected device into `networking.networkmanager.unmanaged`.

I'm wondering if there is any solution to this. Since this triggers for me always. So, I'm not even sure if my `nixos-rebuild switch` does actually complete.

As mentioned in the initial description of the issue:

This never seems to be the case, since the actual message is "Started Network Manager.".
Apart from that I'm wondering why it relies on a log message string, when the status of `NetworkManager.service` would be way less prone to errors.

I think I'm running into this.

`systemd-networkd-wait-online.service`

Aug 22 08:38:49 erin-laptop systemd[1]: Starting Wait for Network to be Configured...
Aug 22 08:40:49 erin-laptop systemd-networkd-wait-online[26249]: Timeout occurred while waiting for network connectivity.
Aug 22 08:40:49 erin-laptop systemd[1]: systemd-networkd-wait-online.service: Main process exited, code=exited, status=1/FAILURE
Aug 22 08:40:49 erin-laptop systemd[1]: systemd-networkd-wait-online.service: Failed with result 'exit-code'.
Aug 22 08:40:49 erin-laptop systemd[1]: Failed to start Wait for Network to be Configured.

It only happens after I've connected my USB-C dock with an ethernet connection at least once after boot. (Note: I'm running tmpfs on root, so my system should "forget" everything about my dock on reboot.)

I don't use NetworkManager so I wouldn't know, but in the case of systemd-networkd there are relevant options under `systemd.network.wait-online`: `anyInterface` and `ignoredInterfaces`. I recommend at least setting the former to `true` on laptops.

Warning about failed units is pretty much the last thing that the activation script does, so it's probably fine (but the failure should be fixed, of course).

BTW I've "fixed" this by setting

nix
# udev 250 doesn't reliably reinitialize devices after restart
systemd.services.systemd-udevd.restartIfChanged = false;

But this is really an upstream systemd bug.

My Ubuntu has:

❯ systemctl cat NetworkManager-wait-online.service
# /lib/systemd/system/NetworkManager-wait-online.service
[Unit]
Description=Network Manager Wait Online
Documentation=man:nm-online(1)
Requires=NetworkManager.service
After=NetworkManager.service
Before=network-online.target

[Service]
# `nm-online -s` waits until the point when NetworkManager logs
# "startup complete". That is when startup actions are settled and
# devices and profiles reached a conclusive activated or deactivated
# state. It depends on which profiles are configured to autoconnect and
# also depends on profile settings like ipv4.may-fail/ipv6.may-fail,
# which affect when a profile is considered fully activated.
# Check NetworkManager logs to find out why wait-online takes a certain
# time.

Type=oneshot
ExecStart=/usr/bin/nm-online -s -q
RemainAfterExit=yes

# Set $NM_ONLINE_TIMEOUT variable for timeout in seconds.
# Edit with `systemctl edit NetworkManager-wait-online`.
#
# Note, this timeout should commonly not be reached. If your boot
# gets delayed too long, then the solution is usually not to decrease
# the timeout, but to fix your setup so that the connected state
# gets reached earlier.
Environment=NM_ONLINE_TIMEOUT=60

[Install]

My latest NixOS config has:

{ after = [ ... ]; aliases = [ ... ]; before = [ ... ]; bindsTo = [ ... ]; confinement = { ... }; conflicts = [ ... ]; description = ""; documentation = [ ... ]; enable = false; environment = { ... }; jobScripts = [ ... ]; onFailure = [ ... ]; partOf = [ ... ]; path = [ ... ]; postStart = ""; postStop = ""; preStart = ""; preStop = ""; reload = ""; reloadIfChanged = false; reloadTriggers = [ ... ]; requiredBy = [ ... ]; requires = [ ... ]; requisite = [ ... ]; restartIfChanged = true; restartTriggers = [ ... ]; runner = error: attribute 'ExecStart' missing

 at /nix/store/6dgpkrc0gxlndr4j2524ihlsr8209ph7-source/nixos/modules/testing/service-runner.nix:65:9:

 64| my $cmd = <<END_CMD;
 65| ${service.serviceConfig.ExecStart}
 | ^
 66| END_CMD
«derivation

based on this definition:

console
 systemd.services.NetworkManager-wait-online = {
 wantedBy = [ "network-online.target" ];
 };

Where is this `ExecStart` coming from in your configurations?

@blaggacao: The reference to `nm-online` comes from upstream service unit `NetworkManager-wait-online.service`, not from nixpkgs itself.

I've been tripping over this bug for quite some time now and it is annoying for users. As mentioned above, the error can be worked around with:

nix
systemd.services.NetworkManager-wait-online.enable = lib.mkForce false;
systemd.services.systemd-networkd-wait-online.enable = lib.mkForce false;

I was concernd if there might be other dependencies or services that require this to be enabled, so I grepped through nixpkgs for both. These are the mentions:

### NetworkManager-wait-online.service
- modules/networkmanager The actual definition of this service
- pkgs.hqplayerd can probably be removed

### systemd-networkd-wait-online.service
- modules/system/boot/networkd Service devinition
- Used in various nixos tests, which should not be relevant to normal operation of the system (?)
 - nixos/tests/systemd-networkd-dhcpserver-static-leases.nix
 - nixos/tests/kea.nix
 - nixos/tests/systemd-networkd.nix
 - nixos/tests/systemd-bpf.nix
 - nixos/tests/systemd-networkd-dhcpserver.nix

### TL;DR

Upon first look the usage of these two services seem minimal to me and they are causing more problems that doing good. Agreeing with @domenkozar's proposal, I'd vote to disable them per default. If this is agreed upon, I can submit a PR

I've been running with that service disabled for 6 months now and have not experienced a single issue. Don't count my voice too heavily, though :wink: ! :+1:

If we're going to work around this I'd still prefer `systemd.services.systemd-udevd.restartIfChanged = false;` as the other workaround just masks the issue while udev's still half-broken.

For some reason, that didn't work for me. On rebuild it said "not restarting service"

Yep confirming what @pinpox said:

updating GRUB 2 menu...
NOT restarting the following changed units: systemd-udevd.service

Still seeing the issue on HEAD. Disabling wait-online like mentioned previously fixes nixos-rebuild.

systemd.services.NetworkManager-wait-online.enable = lib.mkForce false;
systemd.services.systemd-networkd-wait-online.enable = lib.mkForce false;

`NetworkManager-wait-online.service` is a dependency of `network-online.target`:

console
$ systemctl list-dependencies --reverse NetworkManager-wait-online.service
NetworkManager-wait-online.service
● └─network-online.target
● ├─dnscrypt-proxy2.service
● ├─wireguard-wg-intra-peer-[...].service
● └─multi-user.target

console
$ systemctl cat NetworkManager-wait-online.service | grep network-online.target
Before=network-online.target
WantedBy=network-online.target

As others pointed out, the `-s` (aka. `--wait-for-startup`) flag is causing some deadlock:

# `nm-online -s` waits until the point when NetworkManager logs
# "startup complete". That is when startup actions are settled and
# devices and profiles reached a conclusive activated or deactivated
# state. It depends on which profiles are configured to autoconnect and
# also depends on profile settings like ipv4.may-fail/ipv6.may-fail,
# which affect when a profile is considered fully activated.
# Check NetworkManager logs to find out why wait-online takes a certain
# time.

I guess using `-s` is a more accurate notification than simply waiting for `NetworkManager.service` to be up, and I don't know why it sometimes fails to notice that the startup is complete, nor whether removing it has any significant drawbacks, but keeping it causes `switch-to-configuration test` to fail often enough to bother me, so until a more correct fix is available, I'm personally removing it:

nix
{ pkgs, ... }: {
 systemd.services.NetworkManager-wait-online = {
 serviceConfig.ExecStart = [ "" "${pkgs.networkmanager}/bin/nm-online -q" ];
 };
}

Also running into these deadlocks on recent versions of nixpkgs.

This has been happening to me consistently for over a year. The cause seems to be due to what I've described in URL

I used (a simplified version of) @ju1m's workaround, but using the workaround from URL seems to be even better. However, I don't know how to include that in my system configuration rather than running it manually when needed.

My instance of this issue is indeed caused by a wireguard interface, but adding the interface to `networking.networkmanager.unmanaged` (via name or `type:wireguard`) did not change a thing. Having done some more digging online, I believe I know why: prior to udev initialization completing for a link, NM cannot be sure what the name or type of the link will be (as udev may make changes), and so it is *not possible* to filter a device prior to udev marking it as configured.

I'm still looking for a workaround.

I have the issue on the MacBook Pro 2019 16,1 with t2 patches after switching to iwd, because of network issues.

I'm new to NixOS, I have this error as well:

bash
jul 20 19:19:13 nixos systemd[1]: Starting Network Manager Wait Online...
jul 20 19:20:13 nixos systemd[1]: NetworkManager-wait-online.service: Main process exited, code=exited, status=1/FAILURE
jul 20 19:20:13 nixos systemd[1]: NetworkManager-wait-online.service: Failed with result 'exit-code'.
jul 20 19:20:13 nixos systemd[1]: Failed to start Network Manager Wait Online.
warning: error(s) occurred while switching to the new configuration

 I put this line in my configuration.nix (`systemd.services.NetworkManager-wait-online.enable = lib.mkForce false;`), but then I get this error:

bash
$ sudo nixos-rebuild switch --upgrade
unpacking channels...
error: undefined variable 'lib'

 at /etc/nixos/configuration.nix:16:56:

 15| boot.loader.systemd-boot.enable = true;
 16| systemd.services.NetworkManager-wait-online.enable = lib.mkForce false;
 | ^
 17| boot.loader.efi.canTouchEfiVariables = true;
(use '--show-trace' to show detailed location information)
building Nix...

And I can't update anymore, so I am stuck and a fairly recent clean NixOS install. Is there a way to force the update for now? What is the best structural solution?

Thank you!

Looks like `nixos-generate-config` adds this in the beginning of /etc/nixos/configuration.nix:

[...snip comment block...]

{ config, pkgs, ... }:

whereas most people eventually modify that to `{ config, pkgs, lib, ... }:`, providing configuration.nix with the missing `lib` argument.

I think nixos-generate-config should be updated to include `lib` by default. Although the default config doesn't need/use `lib`, it quickly becomes something new NixOS users want to use.

Thanx for the quic reply.

I have added { config, pkgs, **lib**, ... }:

In the first line of my configuration.nix, but I get the same error...

Then please open a new issue or post on URL asking for help.

This issue has been mentioned on **NixOS Discourse**. There might be relevant details there:

 URL

Ok, this solved it for me: URL

Just dit `sudo nixos-rebuild boot --upgrade` and rebooted. I left the lib in the `{ config, pkgs, lib, ... }:`.

Thanx for the support!

So, to summarise, the actual error seems to be that NM waits for startup of some interfaces it thinks it should manage but they're never communicated as up to NM because NM was never supposed manage that interface?

When I say `nmcli`, NM thinks it does not manage my tailscale0 interface:

tailscale0: unmanaged
 "tailscale0"
 tun, sw, mtu 1280

Why would it be waiting on it then? The other interfaces are up and I'm sending you this message via one of them, so it can't be waiting on those.

I'll try to explicitly add it to `networking.networkmanager.unmanaged` and report back after a few rebuilds in a few weeks.

No, `nm-online` is just weirdly broken and locks up if you try to do it on an already running system. It's probably an easy code fix but I'm not familiar with the codebase at all.

Is that something that has to be fixed upstream? If so, could you create an issue for it?

Reading through those. Are we using the mentioned `-s` flag? If so, maybe we can remove it? Running `nm-online -s` manually hangs for me, while `nm-online` (without `-s`) seems to work correctly.

Alright, I'll stick with my workaround then for now. I would like to contribute a real fix for this or sort it out properly, but don't know how to continue.

I had this problem for well over a year, but since I added this to my `configuration.nix` the problem has not reoccurred:

nix
systemd.services.NetworkManager-wait-online = {
 serviceConfig = {
 ExecStart = [ "" "${pkgs.networkmanager}/bin/nm-online -q" ];
 };
};

Nor have I had any other network problems.

Also,

shellsession
$ nm-online -s
Connecting............... 30s [started]
$ journalctl -b -u NetworkManager-dispatcher.service
Jul 10 16:24:41 melchizedek systemd[1]: Starting Network Manager Script Dispatcher Service...
Jul 10 16:24:41 melchizedek systemd[1]: Started Network Manager Script Dispatcher Service.
Jul 10 16:24:51 melchizedek systemd[1]: NetworkManager-dispatcher.service: Deactivated successfully.
Jul 11 17:43:59 melchizedek systemd[1]: Starting Network Manager Script Dispatcher Service...
Jul 11 17:43:59 melchizedek systemd[1]: Started Network Manager Script Dispatcher Service.
Jul 11 17:44:09 melchizedek systemd[1]: NetworkManager-dispatcher.service: Deactivated successfully.
Jul 14 18:41:15 melchizedek systemd[1]: Starting Network Manager Script Dispatcher Service...
Jul 14 18:41:15 melchizedek systemd[1]: Started Network Manager Script Dispatcher Service.
Jul 14 18:41:25 melchizedek systemd[1]: NetworkManager-dispatcher.service: Deactivated successfully.
...
$ last reboot -1
reboot system boot 5.15.119 Mon Jul 10 16:24 still running

`NetworkManager-dispatcher.service` seems to run at various times that aren't related to reboots or `nixos-upgrade switch`

@neilmayhew what does that exactly do? If I understand the man page correctly, `-q` only omits the output, without changing any functionality

Adding `tailscale0` to `networking.networkmanager.unmanaged` on my system that's currently experiencing this issue did not resolve the problem.

Not on the NixOS side; we'd have to fix it upstream I imagine.

Is the problem that `nm-online` with `-s` on an already-up system hangs (what this thread seems to think)?

Is the problem that `NetworkManager-wait-online.service` _should_ not have `-s` (what upstream seems to think)?

I do not see any confusion about this from the upstream side; they did not claim that distros should not use `-s` at any point in the thread you linked.

We want a flag that does what `-s` is currently documented to supposedly do:

 -s | --wait-for-startup
 Wait for NetworkManager startup to complete, rather than waiting for network connectivity specifically. Startup is considered complete once NetworkManager has activated (or
 attempted to activate) every auto-activate connection which is available given the current network state. This corresponds to the moment when NetworkManager logs "startup
 complete". This mode is generally only useful at boot time. After startup has completed, nm-online -s will just return immediately, regardless of the current network state.

 There are various ways to affect when startup complete is reached. For details see NetworkManager-wait-online.service(8).

The manual even explicitly clarifies behaviour after bootup is complete.

This is precisely what we want but `nm-online` simply does not work as documented. Instead of returning immediately, it blocks indefinitely under certain conditions.

Is this related to kernel/firmware upgrade? This problem just occurred on my machine after a `nixos-rebuild switch`, manually invoking `nm-online -s` also waited 30s and timed out, but after a reboot I can no longer reproduce this, even with another `nixos-rebuild switch`. I updated my kernel and firmware in the first `nixos-rebuild switch`, updating from linux 6.8 to 6.9.

I noted that `tailscaled.service` is restarted before `NetworkManager.service` and `NetworkManager-wait-online.service`.

restarting the following units: home-manager-inme.service, home-manager-root.service, nix-daemon.service, polkit.service, systemd-journald.service, tailscaled.service
starting the following units: NetworkManager-wait-online.service, NetworkManager.service, audit.service, bluetooth.service, kmod-static-nodes.service, logrotate-checkconf.service, mount-pstore.service, network-local-commands.service, network-setup.service, persist-\x27-nix-persist-etc-machine\x2did\x27.service, persist-\x27-nix-persist-home-inme-.bash_history\x27.service, persist-\x27-nix-persist-home-inme-.gitconfig\x27.service, persist-\x27-nix-persist-home-inme-.repo_.gitconfig.json\x27.service, power-profiles-daemon.service, resolvconf.service, rtkit-daemon.service, systemd-hostnamed.service, systemd-modules-load.service, systemd-oomd.socket, systemd-sysctl.service, systemd-timesyncd.service, systemd-udevd-control.socket, systemd-udevd-kernel.socket, systemd-update-done.service, systemd-vconsole-setup.service, waydroid-container.service, wpa_supplicant.service, xray.service

Also confirmed in journalctl:

May 28 07:26:58 thinkbook-16-plus-nixos systemd[1]: Started Tailscale node agent.
░░ Subject: A start job for unit tailscaled.service has finished successfully
░░ Defined-By: systemd
░░ Support: URL
░░
░░ A start job for unit tailscaled.service has finished successfully.
░░
░░ The job identifier is 3587.
...
May 28 07:26:58 thinkbook-16-plus-nixos systemd[1]: Starting Network Manager...
░░ Subject: A start job for unit NetworkManager.service has begun execution
░░ Defined-By: systemd
░░ Support: URL
░░
░░ A start job for unit NetworkManager.service has begun execution.
░░
░░ The job identifier is 3744.
...
May 28 07:26:58 thinkbook-16-plus-nixos systemd[1]: Starting Network Manager Wait Online...
░░ Subject: A start job for unit NetworkManager-wait-online.service has begun execution
░░ Defined-By: systemd
░░ Support: URL
░░
░░ A start job for unit NetworkManager-wait-online.service has begun execution.
░░
░░ The job identifier is 3743.

And if I `systemctl stop tailscaled` before `nixos-rebuild switch`, `tailscaled.service` will be started after `NetworkManager.service` is started, and then `NetworkManager-wait-online.service` will exit gracefully.

Big if true.

That feels like something that should be solvable using a few systemd Before/After directives in the right places.

Though tailscaled.service already contains

After=network-pre.target NetworkManager.service systemd-resolved.service

IIRC `After` should be applied in reverse when stopping/restarting.

for me

sh
sudo nixos-rebuild boot --flake .#nixos

instead of

sh
sudo nixos-rebuild switch --flake .#nixos

did the trick..

@atemu could you challenge upstream about the `-s` flag and its documented behavior ?

Just to add a +1 to the tailscale discussion - I did not have this issue at all prior to installing tailscale, and now have it any time a rebuild needs to restart NetworkManager.

It seems currently the switching script (the Perl version) doesn't have such mechanisms, neither does the Rust version. For the Perl script, it just `systemctl restart`s and then `systemctl start`s. Meanwhile the Rust version seems to be waiting all `restart` threads to finish before starting the `start` threads.

A probable solution to this is to use combine these two commands into one `systemctl restart`, as stopped services will just get started on `systemctl restart`, and systemctl/systemd should be able (?) to resolve these Before/After relations if we invoke the command in one go. For the rust version, maybe removing that `block_on_jobs` and changing accordingly will be sufficient.

—— in case this is really the cause of the problem.

Thank you for this observation - adding `systemd.services.tailscaled.after = ["NetworkManager-wait-online.service"]` seem to have solved the issue for me

I'd invite everyone to try this out on their machines and report back.

If this works, that'd be a quick and easy one-line patch to the tailscale NixOS module; fixing this for everyone.

Can confirm this fixes it. @Atemu +1 for including the fix in the tailscale module

Is this only fixed when using tailscaled, for users not using tailscaled, they should apply the workaround mentioned ? Is this really a good solution for new users?

@AkechiShiro have you observed this without using Tailscale?

As far as I understand the issue is tied down to the combination of Tailscale firing up before NetworkManager and depending on it being online, ending up in a deadlock essentially

The same issue happened to me with wireguard in the past. I'll try the suggested fix

Mind posting your configuration.nix in a gist?
We should be able to create a minimal flake and reproduce this with build-vm. Maybe cut out the VM's network access on purpose.

We should reopen this issue then if this happens outside of just Tailscale & NM

I do use Wireguard yes, so it could be I have the same issue as @pinpox

Can anyone confirm whether they still experience this issue without using tailscaled?

As recently as two years ago, I had this issue without knowingly using tailscaled. (Posted to #59603.)
Since then, I'm using the workaround from the OP and have lived happily ever since.

The problem is perhaps that tailscaled is not *the only* service that activates `NetworkManager-wait-online.service`, so it doesn't make sense to fix this issue in tailscaled.

Perhaps this issue has since been fixed by an unrelated patch upstream (however that would happen), but I'm not optimistic. Let me try to get rid of the workaround and see if it is still an issue.

Thanks @sgraf812 I may try to reach out to systemd/udev maintainers about issue #182449 hopefully we can nail this issue down once and for all

We did receive an answer from Poettering but I don't know how to properly answer back, if anyone more experienced could pitch in : URL

This issue has been mentioned on **NixOS Discourse**. There might be relevant details there:

 URL

@Atemu ran into it as well, but already had a running system with tailscale up. Manually shut down tailscale, and rebuild succeeded (and tailscale was back up). Tested with several rebuilds and restarts, works ok so far

An idea I had on this is that we could perhaps hack around this using a systemd unit that `Conflicts` the units which appear to block nm-online.target such as tailscaled. It'd be `Require`d by the nm-online target and we'd then have another target which wants all those units and is wanted by the network-online target but `After` the nm-online target.

Maybe this would help? (I closed because earlier I thought it was obsolete)

And I've made a small flake version for testing purpose: URL

At least last time I used this, I didn't have to `systemd stop tailscaled` beforehand when tailscale was updated.

I can try testing with it.
Think the repro steps are:
1. Have a system with Tailscale and NM-wait-online
2. Disable nm-wait-online with the hack above
3. Put nm-wait-online back in

On the next rebuild it should get stuck. I'll try this later today and report back if it reproduces consistently. Note: the Tailscale fix was backported to 24.05 so need to pin nixpkgs to a commit prior to that

Also still experiencing this with tailscaled on nixpkgs unstable `bc947f541ae55e999ffdb4013441347d83b00feb` (I think that's the right sha.)

If I stop the tailscaled unit I can then rebuild OK, and the service comes back up again.

I'm still running into this problem. However, I'm not using NetworkManager, but systemd-networkd. Adding the following line to my config seemed to fix things:

nix
systemd.services.tailscaled.after = ["systemd-networkd-wait-online.service"];

Perhaps we should also add that line to the tailscale service definition?

Well, I haven't been using it for a while as I am no longer experiencing this problem on the unstable branch until `a73246e2eef4c6ed172979932bc80e1404ba2d56`. Which commit are you on now?

That flake can be used by making an overlay over nixpkgs, disable `system.switch.enable` and enable `system.switch.enableNg` in the `configuration.nix`.

Are the only current workaround is:

nix
# /etc/nixos/configuration.nix
systemd.network.wait-online.enable = false;

Hitting the same issue here. Adding `systemd.services.tailscaled.after = [ "systemd-networkd-wait-online.service" ];` fixed it for me too.

`systemd.services.tailscaled.after = ["NetworkManager-wait-online.service"]` stopped working for me today; just had a switch fail on me despite verifying that NetworkManager-wait-online.service was listed in the systemd override file. The earlier workaround removing `-s` from the ExecStart continues to work fine.

I'm highly suspicious that this has to do with NetworkManager being updated; I'm not sure what dependency caused a rebuild in this particular case, but the NetworkManager hash did change for me between generations. That might be the trigger to actually reproducing this.

I am also hitting this issue on `25.05` despite using

nix
 systemd.services.NetworkManager-wait-online.enable = false;
 systemd.services.tailscaled.after = [ "systemd-networkd-wait-online.service" ];


Question: 1.4 Hangs server on Version checking?

You need to be more specific than "my save game was corrupted". Can you not enter your world anymore with a warning about safe mode or are there things like chunk errors?

1.4 Hangs server on Version checking
Recently updated to 1.4 and now the server hangs during Neoforge version checking stage. Reverting this mod to the last version fixed the issue.

Here was the last log before I reverted back to 1.3.10
debug-5.log.gz

I take a look at the log it's the same as #23.
Instruction: Wait for new update from CTOV!

CTOV incompatible with 1.4
updated today adn minecraft suddenly said my save game was corrupted. did some trial and error and found out it was the new version of litho that caused this

I had a the same thing happen to me. Wouldn't load world saying it was corrupted, going back to previous version fixed the issue.
It also caused Minecraft to crash completely when clicking on Singleplayer with no saved worlds or deleting the last world in the single player list. Here are some crash reports for both scenarios.
crash-2025-01-09_19.53.40-client.txt
crash-2025-01-09_19.57.29-client.txt

CTOV is incompatible with Lithostitched 1.4 until they update to a new format for some internal files.

Yup, I can confirm this crash. I have been trying to fix this problem for 2 hours, and found out that
When I disabled (lithostitched-neoforge-1.21.1-1.4) and I revert to v1.3.10 the crash does not happen.

So because my particular world depends on the CTOV, I will revert back to v1.3.10 of Lithostitched,
until CTOV updates their mod. 2 hours of frustrations trying to figure out what it was. My head hurts. 🤦♂

Lithostitched Crash

A page on the wiki has now been made for that. It's a small change for consistency with other worldgen modifiers and I have no plans for future breaking changes right now.

CTOV has updated to Lithostitched 1.4. Issues with other mods not working with 1.4 should be moved to their own issue page(s).

@Apollounknowndev Why did you delete my comment? I was simply wondering if your project follows semantic versioning so I can watch out for these breaking changes in the future.

Semver will be followed going forwards. In the future, please keep unrelated questions in their own reports.


Question: Hervorhebungen ausschalten?

In Player 1.28.3 habe ich den gegenteiligen Wunsch implementiert!
Seid ihr sicher, dass ihr wieder zurück wollt?

Hervorhebungen ausschalten
Wir bräuchten für die Hervorhebungen eine "Ausschalten"-Möglichkeit. Unsere Idee ist es, beim Auswählen der zu hervohebenden Stelle (z.B: beim Trigger) im Dropdown-Menü noch eine "Hervorhebungen ausschalten"-Option hinzuzufügen.

Ich habe ein älteres Ticket gefunden, dass sehr ähnlich klingt: #516
Kann eins davon zu?

Ja, ich habe das alte Ticket geschlossen. Dieses Ticket hier hängt aber auch mit URL zusammen - beide Tickets sollten am besten zusammen bearbeitet werden.

mehrere Hervorhebungen gleichzeitig
Wäre es möglich bei Triggern und normalen Hervorhebungen die Möglichkeit zu geben, mehrere Hervorhebungen gleichzeitig zu aktivieren?

Jein - also ja, wir sind uns sicher, dass wir es wollen, aber nur wenn wir weiterhin eine Funktionalität haben auch mit Triggern Hervorhebungen zu überschreiben / auszuschalten (siehe URL


Question: New re-verify endpoint specifications?

During a recent call, we raised the question of what happens to contracts verified with a bugged Sourcify version when the same contract is later inserted with a major (correct) version. Someone suggested periodically pruning the database, but we realized this could be a problem for our Parquet exports: we cannot edit parquet exports already created, so bugged sourcify contracts will be included in the exports.

However, I don’t think this will actually be a problem. When importing contracts from a Parquet export, users can filter out contracts verified with bugged Sourcify versions, or replace them once a version with a major fix appears in a subsequent export file.

The open questions are:
* Does it make sense to prune?
* When should we prune?
* Should we also prune the VerA database?

New re-verify endpoint specifications
We often need to fix contracts data due to misaligned data. Currently, we lack a generic method to replace an existing misaligned contract in our database, so we’ve been forced to create custom scripts to address specific issues (#1974).

In this issue, I propose introducing a new private endpoint—callable only by the Sourcify instance maintainer—that allows replacing an existing contract. To improve the performance of this endpoint, we can optimize it by skipping compilation and RPC-based on-chain data fetching, instead relying on information already stored in the database. Two properties, getCompilationFromDatabase and getOnchainDataFromDatabase, can be used to specify that the data should be retrieved from the database.

### `/private/replace-contract`

### Parameters
- **chainId**
- **address**
- **creation_txhash**
- **getCompilationFromDatabase: bool**. If set to true it skips compilation and queries compilation information from the database
- **[if(getCompilationFromDatabase==false)] jsonInput**: optionally pass jsonInput, if it's passed it uses the passed one. if it's not passed it uses the jsonInput stored in the database (but it will recompile to get the jsonOutput)
- **getOnchainDataFromDatabase: bool**. If set to true it skips calling rpc to find onchain information and queries onchain information from the database

### Process
1. fetch onchain information from the database if specified in the request's parameter, otherwise call the rpc
2. fetch compiled information from the database if specified in the request's parameter, otherwise compile the contract
3. call lib-sourcify verification flow
4. if successful
 4.1. delete existing `sourcify_match` and all linked rows in foreign tables, only if those rows are dangling. If there are non-dangling rows then return an error and we manually have to evaluate the case.
 4.2. insert the new verification
5. if not successful, return an error

### lib-sourcify refactoring

Suggestion from @manuelwedler: we could implement a new class in lib-sourcify `DatabaseCompilation` that follows the `AbstractCompilation` interface and gets the compiled bytecode and compilation artifacts as constructor arguments. That could be useful for skipping the compilation and avoiding the monkey patching we do at the current `upgradeContract` endpoint.

We an potentially do something similar for `SourcifyChain`.

Note that the `verification_jobs` table references the `verified_contracts` table. Due to that, you won't have any dangling rows after deleting a `sourcify_match` (for contracts verified with API v2).

Should `DatabaseCompilation` be in the lib-sourcify or rather in sourcify-server next to this private endpoint, but following the `AbstractCompilation` spec?

Imo, it could also be in lib-sourcify. This functionality might also be interesting for others.

Currently, we have this contraint on `verification_jobs` table:

CONSTRAINT verification_jobsverif_ied_contract_id_fk FOREIGN KEY (verified_contract_id) REFERENCES verified_contracts(id) ON DELETE RESTRICT ON UPDATE RESTRICT

This means it will prevent to delete the `verified_contracts` row, if it is still referenced inside `verification_jobs`.
Generally, I think it is fine when the `verification_jobs` row does not reference any `verified_contract`, but we would need to adapt the constraint.

I also wonder what we do with VerA in this endpoint. I guess this is only for the Sourcify database. It still feels a bit inconsistent to not update VerA to me.

Mmmh, so you’re thinking of a generic `ConstructorCompilation` (meaning we pass everything through the constructor), not something specifically tailored to our database? In that case, I think it makes sense.

My original idea—which I now realize I didn’t actually write down—was to have a `DatabaseCompilation` built specifically for our database. That way, we could just pass the address and chain, and it would fill in all the details for us. The two proposals are actually compatible by having a `DatabaseCompilation.createCompilation` similar to `SolidityMetadataContract.createCompilation` that creates a `ConstructorCompilation`.

I think this falls under the topic of #2089

Exactly. Just take a standard json output as constructor parameter. We can already easily generate the standard json output from the database.

I agree. That also makes sense.

Yes, we would need the verifier version column. Still, wanted to raise that topic. Would you update the endpoint once we have the verifier version, and add the functionality to upgrade contracts in VerA?

I don't like the name `DatabaseCompilation`, esp. for lib-sourcify. What we want to do is kind of "mocking" the `compile()` (so maybe `MockCompilation` ?) method of the `...Compilation` classes, given we have a compiler output. This compiler output can come from the database or anywhere. In our private endpoint this would be fetched from the DB and formatted accordingly.

Similarly the "`MockVerification`" would have its bytecodes and other onchain info filled when constructed, but will have the same interface as `Verification`. We're probably talking about the same thing but just wanted to make it clear.

Yeah actually let's make it a generic compilation with required fields (we should do some validation), and it should be straightforward to format an output from the DB to that generic format.

Yes, totally right. We should not change the order of bytecode fetching, then compiling. I missed that when I made my proposal.

The only solution I see to this is also separating bytecode fetching into its own class as suggested in URL We could keep the order of first bytecode fetching, second compiling, and only pass the results of both to the `Verification` class.

The other points of your comment seem manageable / fine to me.

Nevertheless, my proposal seems like a quite big refactoring. I tend to say just go with the original idea. If you still see value in my new proposal, we can still target it later. Otherwise, just scrap it.

Find a solution for UNIQUE constraint blocking re‑verification of contracts originally verified by a buggy Sourcify server
Copying the comment: URL

### Root Cause
- Our general Vera policy is “**INSERT only, never UPDATE**”.
- We didn’t anticipate a scenario where a **newer** verifier could succeed on a contract that an **older** version failed.

### Current state of the affected contracts
- The values in `verified_contracts.compilation_id` and `verified_contracts.deployment_id` are correct.
- The old, bugged Sourcify server simply couldn’t find the creation match.

### Problem
- When we re‑verify these contracts with the latest Sourcify's server, it tries to INSERT the new `verified_contracts` row (with the creation match) with the same `compilation_id` and `deployment_id`.
- A **UNIQUE** constraint on `(compilation_id, deployment_id)` prevents this, so the upgrade—fails.

### Proposed solution
1. **Add** a new column:

sql
 ALTER TABLE verified_contracts
 ADD COLUMN verifier_version TEXT NOT NULL;

2. Extend the UNIQUE constraint to include this column.
3. Populate verifier_version with, for example, "sourcify@1.0.0" (or similar).

This lets us INSERT a fresh row whenever a newer verifier succeeds on a previously failed contract.

Well, looking at this again why don't we add "runtime_match" "creation_match" "runtime_metadata_match" "creation_metadata_match" to the unique constraint?

It's not enough to add these fields. We would have to add also transformations to the constraints (since they can change between different sourcify versions), I don't even know if it's possible and if it makes sense...

I think I prefer the first option because keeping the "INSERT only, never UPDATE" policy can gibe us some benefit for the parquet export. We could work on that for having append only export.

The issue I see with the verifier version column is that we need to check all columns and only insert if something changed (apart from the verifier version).

We haven't commited ourselves to maintaining immutable (append only) databases yet. It was just a suggestion.

IMO pruning at this stage makes sense. I wouldn't expect users to be able to correctly filter out contracts with buggy Sourcify versions.

Since our DB is not extremely large at the moment, it should be ok to modify data. It would require people to re-download everything and re-import, if they want to instantiate a DB.

We should regardless have an idea to how long it takes to download everything and to initialize a DB instance. We also need instructions and maybe a script for people to do this. The issue already exists: #2038 Maybe something for the next sprint.

Finally, I think over time we can find a compromise between a mutable database vs. an append only DB. We should look how people in general such a problem. Maybe some kind of versioning of the exports etc.