Question: `ivy.flatten`?

Hello @Viditagarwal7479
Thanks for the suggestion but we have this function on Ivy API ivy.max()🙂

`ivy.flatten`
ivy codebase lacks ivy.flatten method to flatten a matrix or a tensor which I feel is relevant for a lot of frontend/backend methods that would be added in future

Hi @5hv5hvnk,

Thanks for flagging, I will mention to the team!

I will leave this issue open and report back :)

Hi,

I have linked the issue to a PR that was working on an implementation for `ivy.flatten`. Once this PR has been merged this issue will be closed.

Thanks for Flagging!

We have discussed internally, and we have accepted that this function should be added to Ivy's functional API. This item has been added to our ongoing Ivy Extension issue. Therefore, this issue will now be closed. Thanks!

Extend Ivy Functional API
Extend Ivy functional API ivy//ivy/functional/experimental. Each function will need implementation in each backend API and also the Ivy API. The tests will also need to be implemented. See the Deep Dive for more general information on how the functions should be implemented. Particular attention should be placed on the Superset Behaviour section when considering how best to design the function, and what features and arguments it should have. More information about this specific task is provided in the Open Tasks section of the contributor guide. All the implementations have to be placed in `experimental` directories for both, functional API, backend APIs, Array, and Container instances.

`Linear algebra functions:` ivy//ivy/functional/experimental/linalg.py

- [x] #5991
- [x] #6180
- [ ] #6269
- [x] #6228
- [x] #7748
- [x] #8990
- [ ] eigh_tridiagonal
- [x] #8806
- [ ] solve_triangular
- [ ] #9579
- [ ] #9325

`sorting functions:` ivy//ivy/functional/experimental/sorting.py

- [x] #6138

`Manipulation functions:` ivy//ivy/functional/experimental/manipulation.py

- [x] #5542
- [ ] #5944
- [x] #5787
- [x] #5614
- [x] #5683
- [x] #5977
- [x] #6874
- [x] #5978
- [x] #6079
- [x] #7967
- [x] #6787
- [x] #6334
- [x] #6340
- [ ] frombuffer
- [x] #7222
- [x] #6873
- [x] #6222
- [x] #6734
- [x] #6728
- [x] #6761
- [x] #6883
- [x] #6886
- [x] #6908

`Random functions:` ivy//ivy/functional/experimental/random.py

- [x] #6156

`Statistical functions:` ivy//ivy/functional/experimental/statistical.py

- [ ] #7717
- [x] #5715
- [x] #6808
- [x] #6702
- [ ] #9566
- [ ] #7014
- [x] #6934
- [x] #9298
- [ ] histogramdd
- [ ] bincount
- [ ] #9417
- [ ] percentile

`Layers functions:` ivy//ivy/functional/experimental/layers.py

- [x] #5303
- [x] #5548
- [x] #6888
- [x] #5465
- [x] #5541
- [ ] idct
- [x] #9132
- [x] #5620
- [x] #5540
- [x] #5501
- [ ] #8827
- [ ] #5325
- [ ] #5105
- [x] #5147
- [ ] dropout3d
- [ ] collapse_repeated
- [x] #6120
- [ ] inverse_mdct
- [ ] inverse_stft
- [ ] inverse_stft_window_fn
- [ ] mdct
- [ ] quantize
- [ ] adaptive_avg_pool1d
- [ ] #9522
- [ ] #9381

`Elementwise functions:` ivy//ivy/functional/experimental/elementwise.py

- [x] #4236
- [x] #5438
- [x] #4789
- [x] #4237
- [x] #4815
- [x] #6280
- [x] #5793
- [x] #5880
- [x] #5706
- [x] #5716
- [ ] #9255
- [x] #6864
- [x] #6345
- [x] #9208
- [x] #6462
- [x] #6123
- [x] #6810
- [x] #7809
- [ ] frexp
- [x] diff
- [x] #6196
- [x] #9009
- [x] #6482
- [x] #6284
- [x] #6534
- [x] #6468
- [x] #6910
- [x] #6684
- [ ] hypot
- [x] #6350
- [x] #6097
- [x] #6335
- [ ] #9260
- [x] #7150
- [ ] real
- [x] #9171

`Sparse Array class:` ivy//ivy/functional/experimental/sparse_array.py

- [x] #4097

`Activation functions:` ivy//ivy/functional/experimental/activations.py

- [x] #4529
- [x] #8962

`Creation functions:` ivy//ivy/functional/experimental/creation.py

- [x] #6007

`Norms:` ivy/ivy/functional/experimental/norms.py

- [x] #9549

`Utility:` ivy/ivy/functional/experimental/utility.py

- [ ] #6695

max() function is also missing could this be added in active tasks? Then I shall create a PR corresponding to it.
As this is also needed for PyTorch frontend open task URL

Hey @hirwa-nshuti
Ohhh! I see my bad. Later noticed and fixed the error which was coming due to some sort of linking issue :smile:

Hey @hirwa-nshuti
Ohhh! I see my bad. Later noticed and fixed the error, which was coming due to some sort of linking issue smile

Added nanmin and nanmax function for PyTorch, Numpy, Jax, TensorFlow and Paddle backend. Additionally, implement api functions for ivy array and ivy container. #16328 #16327

Hi @YasCoMa, the two functions you are referencing seem to have been reserved already. Please make sure you follow our ToDo issues policy explained here every time you intend to start working on a function.

@khushi-411 Hey, Kushi . I have an idea in mind about implementing `erfc` for `ivy` function API but you're tagged in-that hence, pining you in-here .

Also, I see that you've already made a **PR** URL but it is showing as closed!

In case you're allowed to work with other people or just want to discuss it ; feel free to ping me !
Writing it in here as I can't find you on discord!

Hey, @akshatvishu! My PR is still open, just that I was on leave for a couple of days, so I kept it draft (i.e., work in progress). I'll make sure to finish that soon. Thanks for your kind interest!

If interested, please feel free to pick any other exciting open issues at Ivy! Thanks.

@johanjino

I created an issue on GitHub for the NCE loss function implementation. Could you link the NCE issue with the latest comment on this page?

Here is the link for that issue page

Here you can find instructions on how to link your issues to our ToDos URL

Hey, @hirwa-nshuti I don't see the API specification of modf in docstring, is there any other doc that can help me write the Extended Ivy Functional API of modf.

But i saw and follow the instructions, it was not implemented even in experimental folder. I do not see a reason to disqualify my contribution.

Hi! You can open an issue for this. Please refer to the ToDo List Issues instructions.

As I see you have added them to comments here above, they'll be assigned to you automatically🙂

I have to add a small description in ivy\ivy\functional\ivy\experimental\set.py for the function, i'll create a PR after that, can you assign the task back again, Tysm.

@AnnaTz @hirwa-nshuti the implementation of log_poisson_loss is implemented wrong and tests are failing for it..can u assign that to me ?
I will implement that in the correct way.

my comment to work on binary cross entropy has vanished? Is someone already working on it? @hirwa-nshuti

why I am not getting assigned to binary cross entropy? @hirwa-nshuti

Hey `index_add` is missing from this list and here is an issue explaining why it's needs to be added #26801

Kindly assign the below issue to me.

local_response_norm #27187


Question: No connection?

Can you please verify that the Recorder is actually running?

$ ps ax | grep recorder

and can you see if you can query it from the command line of your Raspi?

$ curl URL

As to the configuration you created in `configuration.yaml`: did you originally add an email address for Let's Encrypt to be activated?

No connection
Hi,

I just installed Owntracks on my server yesterday. The server installed is not exposed to the Internet directly but behind a firewall and a reverse proxy.
On installation I didn't state a mail address to disable let's encrypt and configured my reverse proxy to do the SSL things as well as receive the (anyway optional) traffic on http/https and Forward it to my owntracks server.
The MQTT port 8883 is forwarded directly.
I can access the frontend via Internet and my configured domain.

My problem is, that the app cannot connect to the app. In the firewall I see the traffic is going through. On the server I don't see anything listening on 0.0.0.0:8883 using `ss -l`.
There is just something listening on 127.0.0.1:1883 which turned out to be the mosquito broker. I am now unsure what should be and what the issue may be.
All of the three services are running.
Btw: when I change to HTTP connection instead of MQTT nothing changes.
Reading the MQTT messages on the server I just see the ones from otrec.

Any ideas?

Nothing is listening on the MQTT TLS port (`0.0.0.0:8883`) because you disabled Let's Encrypt and we can thus not configure it for the Mosquitto broker which is now listening on `:1883` as you've already determined.

If you configure your devices to use non-TLS MQTT to that port, and your firewall is correctly configured to redirect to your OwnTracks server you should see data, also log entries for Mosquitto.

OK that works. I see the data in the mosquitto broker now after I reconfigured the mosquito config to listen on 0.0.0.0:1883.
There is also an option to use WebSockets. Would that help? My expectation would be, that in that case the data is send via https to the server, which would be my reverse proxy. That one will "unpack" the https and forward the http to the OwnTracks server. Does it works that way (in theory)? Because in practice this is not working the same way I do it for a couple of regular websites.

Our OwnTracks Recorder can also have positions sent to it via HTTP (not Websockets), and our apps (iOS and Android) support HTTP as well.

Do note, however, that a) support for multiple Friends is more difficult with HTTP and b) many of us think MQTT (over TLS) is the more reliable protocol to use with our apps.

I'm just thinking about the best method I have that is in best case encrypted.
I cannot use let's encrypt behind the reverse proxy because the reverse proxy is already doing it and is handling all of that requests. So the let's encrypt handling will never reach the OT server.
If I would use HTTP(S), the reverse proxy can handle the SSL certificate.

Another thing is: Should the position be visible in realtime in the "Frontend"-map? I see my position in the live map (if something is sent since I opened it). At the moment it is empty for me.

when Frontend loads new data, the positions will be visible, but it doesn't refresh by itself.

I see my position in the live map (if something is sent since I opened it). At the moment it is empty for me.

that's unclear to me: what is empty?

Verify that data is being received and stored by Recorder by checking files in /var/spool/owntracks/recorder/store/rec/ / /YYYY-mm.rec

That's the data the Recorder API uses to populate the maps.

OK, that was a bit confusing:
I see myself on the live map but I see nothing on the "Frontend" map - even after refresh.
In the file of the recorder are several coordinates with the current timestamp.

Please check the following:
1. that you've selected a generous timeframe in Frontend
2. also that you see a list of users / devices (right hand top of Frontend)
3. do you see a version number of owntracks/recorder in Frontend? What does it say?

Hi,
1. I use the standard timeframe (1 month from now to the past)
2. In the right combobox I just have "Show all"
3. It says

[owntracks/frontend] (2.15.3)
[owntracks/recorder] (Loading version...)

4. When I open the Frontend page, the page is always fully zoomed in the sea at the south of Ghana. I think this is just the "middle" of the map and the default if there is nothing else to focus on?

Sorry you're having trouble with this. I fear that during our last release of the Recorder we got the startup script wrong, at least that's what it's looking like.

Please start reading the steps in this issue and see whether they apply to your situation as well. In particular:

1. is the Recorder actually running and providing data
2. is the `baseUrl` in Frontend's config correct?

Hi,
1. Yes, it is running
2. Yes and no. The URL was http:// but it actually is https:// due to the reverse proxy. If you access http:// you will automatically redirected to URL But the Reverse Proxy itself is talking with HTTP (no S) to the OwnTracks server.
It was like `baseUrl: " URL and I added the `s` and restarted the server after. I don't know if that should be enough.

ownTracks frontend without OpenStreetMaps
I followed installation guide: URL and successfully installed all necessary components on a raspberry pi 3 B rev. 1.2

_system info
Last bootstrap: 2024-08-16T11:21:02Z
Ansible version: 2.17.3
OS distro: Debian / 12
OS distribution: bookworm_

"Live map" and "device table" work perfectly, mobile device reports location correctly.

**BUT**, when using "Frontend" instead of OpenStreetMaps just a light blue map is shown -> screenshot
owntracks/frontend (2.15.3)

Any idea?
ownTracks_Frontend

That sounds a bit as though Frontend hasn't yet "seen" data from the Recorder API...

On the top right of Frontend is a selector for users & devices: do you see any data populated there?

And if a bit further on the left you select an older time range?

ownTracks_Frontend_info
that's all I see. Maybe some issues with user permissions? Version is never displayed owntracks/recorder **(Loading version...)** when clicking on "information"
ownTracks_Frontend_info

- yes, recorder is running
- curl URL -> {"version":"0.9.8","git":"0.9.8-0-g488eabe3e3"}
- yes, I originally added an email-address; Let's encrypt SSL certificate is present.

Please check whether the content of Frontend's configuration makes sense, particularly the `baseUrl` value:

$ cat /usr/share/nginx/html/owntracks/frontend/config/config.js

(no need to paste that here, for privacy; if you're unsure, please send us the file to `support@owntracks.org` and we'll look at it)

If the file does make sense, and the URL is accessible *from* your Raspi to your Raspi (`curl` will tell you that), then I'm going to do something unusual and ask you to please reboot the Raspi ...

To clarify: the `baseUrl` must be accessible from your Web browser.

Thank you very much for your hints. Finally I fixed the issue. 🤗

Problem was: I had to open another port than 443 at my ISP router forwarded to raspi 443.

I had to add the opened WAN port to `baseUrl` to get OpenStreetMaps working on Frontends page.

Maybe a feature for quicksetup would be to choose the local port at setup.

Glad you got it working!

So in essence, your ISP offered you, say, TCP port `1009` which you are redirecting into `443`, so you've configured your `baseUrl` to be ` URL for an _nginx_ on the Quicksetup machine running on 443.

I'll make a mental note to consider your feature request. However, one of the ideas behind _quicksetup_ was to keep it as simple (and thus configuration-free) as possible...

Please keep in mind for your own installation, that whenever you re-run `./bootstrap.sh`, say to add a friend, or reconfigure something, that your Frontend's `config.js` will be overwritten.

yes, absolutly correct and I keep in mind that each time I do a re-run of `./bootstrap.sh`, I will have to reconfigure `config.js` to fit my needs regarding ISP WAN/LAN forwarding. Thanks again for pointing me to the problem!


Question: [v2 QUESTION]: Ray "Failed to start the dashboard"?

@kolcker Could you try the following wheel depending on your Python version and see if it works for you:

 URL

 URL

 URL

 URL

[v2 QUESTION]: Ray "Failed to start the dashboard"
**What are you trying to do?**
I'm a beginner for chemprop and machine learning. I recently started trying out the hyperparameter optimization feature. After installing the ray package, I attempted to run the command "chemprop hpopt --data-path --task-type --search-parameter-keywords --hpopt-save-dir ". However, I always encountered errors"Failed to start the dashboard , return code 3221226505". It seems to be a problem with Ray.
ray 2.46.0
chemprop 2.1.2
grpcio 1.71.0
aiohttp 3.11.16
protobuf 6.30.2
Previous attempts

**Previous attempts**
I tried to uninstall ray and used "pip install ray[default]". However, the problem is still there.

**Screenshots**
Image

Can you share this file?

C:\Users\LENOVO\AppData\Local\Temp\ray \session_2025-05-30_09-00-17_586800_104820\logs\dashboard.err

I think it has more details about the error.

@Alang-chem I think it's this error: URL

Which you may be able to resolve by running `pip install 'ray[tune]==2.44.1'` - could you try that and then re-run your command to see if it works?

Maybe your error is similar to this one? URL
Try using these flags to tell ray what resources it can use:

--raytune-use-gpu
--raytune-num-cpus 20,
--raytune-num-gpus 1
--raytune-max-concurrent-trials 1

See also: URL

Thank you for your advice.
I used those command as you said.
run`chemprop hpopt --data-path tests/data/regression.csv --task-type regression --search-parameter-keywords depth ffn_num_layers message_hidden_dim --hpopt-save-dir results --raytune-use-gpu --raytune-num-cpus 20 --raytune-num-gpus 1 --raytune-max-concurrent-trials 1`
I always received the same error
`[W socket.cpp:697] [c10d] The client socket has failed to connect to [LAPTOP-5AHFJTSJ]:55660 (system error: 10049 - �����������У�������ĵ�ַ��Ч��)`
It seems that pytorch canont strat the distributed training in my windows. I find some similar problems on #80638 and #42831. Is it related to Windows 11 system?

Here is my result and error file.

result 25.6.9.txt
error.txt

Seems like there are bits and pieces of this problem reported all over the place.

My suggestion would be to set `USE_LIBUV` to `0` to try and force PyTorch to use a different distributed backend, which is possibly at least tangentially related to this problem.

Other good solution is to just keep downgrading `ray[tune]` and `pytorch` until you find some set of them that works.

This does not seem to be a chemprop error, so I will close this issue for now. Please re-open if the errors start coming from something to do with chemprop, or if you find a solution that might be useful to others.

[Core] `ray.init()` and `ray start` fails on Windows 11 in ray 2.45+
### What happened + What you expected to happen

Hello,

after installing latest release 2.45 - no upgrade, removed older release before - I notice following fundamental issue.

Calling ray start --head in CLI leads to following console output.

I am surprised that the app says "Ray runtime started", but I do not get` ray status` feedback.

Content of `dashboard.log` file

Content of `dashboard.error` file

### Versions / Dependencies

ray == 2.45
Windows == 26100.3915

### Reproduction script

`> ray start --head`

### Issue Severity

High: It blocks me from completing my task.

I'm experiencing the same issue. Some addtional information from raylet.log

[2025-05-04 12:06:23,685 I 32292 25604] (raylet.exe) agent_manager.cc:83: Agent process with name dashboard_agent exited, exit code 1.
[2025-05-04 12:06:23,685 E 32292 25604] (raylet.exe) agent_manager.cc:87: The raylet exited immediately because one Ray agent failed, agent_name = dashboard_agent.
The raylet fate shares with the agent. This can happen because
- The version of `grpcio` doesn't follow Ray's requirement. Agent can segfault with the incorrect `grpcio` version. Check the grpcio version `pip freeze | grep grpcio`.
- The agent failed to start because of unexpected error or port conflict. Read the log `cat /tmp/ray/session_latest/logs/{dashboard_agent|runtime_env_agent}.log`. You can find the log file structure here URL
- The agent is killed by the OS (e.g., out of memory).
[2025-05-04 12:06:23,686 I 32292 35296] (raylet.exe) main.cc:307: Raylet graceful shutdown triggered, reason = UNEXPECTED_TERMINATION, reason message = dashboard_agent failed and raylet fate-shares with it.
[2025-05-04 12:06:23,686 I 32292 35296] (raylet.exe) main.cc:310: Shutting down...
[2025-05-04 12:06:23,686 I 32292 35296] (raylet.exe) accessor.cc:515: Unregistering node node_id=2547f168cb62f79248f1ebdb4d408b1f1ac93d33f1d41244737eb966
[2025-05-04 12:06:23,689 I 32292 35296] (raylet.exe) accessor.cc:528: Finished unregistering node info, status = OK node_id=2547f168cb62f79248f1ebdb4d408b1f1ac93d33f1d41244737eb966
[2025-05-04 12:06:23,689 W 32292 19800] (raylet.exe) store.cc:368: Disconnecting client due to connection error with code 2: End of file
[2025-05-04 12:06:23,695 I 32292 35296] (raylet.exe) agent_manager.cc:116: Killing agent dashboard_agent, pid 8496.
[2025-05-04 12:06:23,695 I 32292 35296] (raylet.exe) agent_manager.cc:116: Killing agent runtime_env_agent, pid 6348.
[2025-05-04 12:06:23,696 I 32292 9940] (raylet.exe) agent_manager.cc:83: Agent process with name runtime_env_agent exited, exit code 1067.
[2025-05-04 12:06:23,697 I 32292 35296] (raylet.exe) io_service_pool.cc:48: IOServicePool is stopped.
[2025-05-04 12:06:23,915 I 32292 35296] (raylet.exe) stats.h:120: Stats module has shutdown.

Updating of grpcio version to 1.72 doesn't help.
Current environment works fine with ray 2.44.1.

**Versions / Dependencies**
ray == 2.45
Windows == 26100.3915

**Issue Severity**
High: It blocks me from completing my task.

Let me know if any additional information will be helpful.

@kolcker: Thanks for your contribution to the error defintion. Changed the title of the GH issue accordingly.

@masoudcharkhabi should this not include the "bug" label as well? It prevents Ray from starting altogether on Windows.

The issue is with redirecting stderr to a file - I'm guess this is closed somewhere. Possibly in PR 51731 - @dentiny ?

*push* - I would consider this topic as a release blocker for windows users. At least, it would be great to get to know a workaround, if applicable.

CI runs on windows succeed, so there is something different in your local machine setup. Could you modify `C:\Users\Philipp\anaconda3\envs\py311-raynew\Lib\site-packages\ray_private\logging_utils.py` to try, except an error around the line `sys.stderr = ...` and print the values of `stderr_fileno`, `stderr_filepath`, `rotation_bytes`, `rotation_backup_count`?

hi @mattip ,

I added strategic logging and got the following insights.

- stdout file number is 1, stderror file number is 2.
- Rotation bytes are always 0, backup count are always 1.
- After complete hard `ray stop` and no ray process running at all, the log rotation init works for stdout, but not for stderr.
- dashboard.out is opened as fully empty file (0 bytes). dashboard.err gets 1 initial byte which is shown in Windows Cmd as "".

See console log.

For your information, in ray version 2.44.1, the same spot looks like this:

It looks like, I localized error even without ray.init.

from ray._private.utils import open_log
from ray._raylet import StreamRedirector
import sys, logging

# Setup redirection for stdout and stderr.
stdout_filepath = 'e:/projects/ray_test/stdout.log'
stderr_filepath = 'e:/projects/ray_test/stderr.log'

print(sys.stdout)
print(sys.platform)

if stdout_filepath:
 StreamRedirector.redirect_stdout(
 stdout_filepath,
 0, #args.logging_rotate_bytes if sys.platform != "win32" else 0,
 1, # args.logging_rotate_backup_count if sys.platform != "win32" else 1
 False, # tee_to_stdout
 False, # tee_to_stderr
 )

stdout_fileno = sys.stdout.fileno()
print("stdout_fileno", stdout_fileno, file=sys.stderr)
print("print to stdout")
sys.stdout = open_log(stdout_fileno, unbuffered=True, closefd=False)

Based on my investigations problem inside `StreamRedirector.redirect_stdout, StreamRedirector.redirect_stderr`. As soon as these methods called we are receiving bad descriptor for writing to log files.

Here is an output:

E:\projects\ray_test\.venv12\Scripts\python.exe E:\projects\ray_test\test.py
stdout_fileno 1
Traceback (most recent call last):
 File "E:\projects\ray_test\test.py", line 25, in
 print("print to stdout")
OSError: [Errno 9] Bad file descriptor
 ' mode='w' encoding='utf-8'>
win32

Process finished with exit code -1073740791 (0xC0000409)

Most likely regression issue appeared somewhere around URL

Please let me know if any information from debugger will be helpful.

@mattip please take a look.

Further finding: The issue does not occur on Windows 10 (Version 10.0.19045 Build 19045).

Here, ray 2.45 shows following behavior for the StreamRedirection:

## New Contributor - Same Windows Issue Confirmed

Hi, I'm new to Ray development and encountered the same Windows issue while following the official development setup guide.

### My Experience:
- **Background:** First-time Ray contributor trying to set up development environment
- **Following:** URL
- **OS:** Windows 11 with conda environment

### Issues Encountered:
- **Ray 3.0.0.dev0 (nightly):** Failed with import errors and version compatibility issues
- **Ray 2.45+:** Similar initialization problems as reported in this issue

### Working Solution:
After reading this thread, I downgraded to Ray 2.44.0, and everything worked as expected — no import errors, and ray.init() runs successfully.
Just sharing my experience in case it helps other Windows users or maintainers trying to reproduce the issue.

bash
pip install "ray[default]==2.44.0"
git checkout ray-2.44.0
python python/ray/setup-dev.py -y

I am able to reproduce this error,

powershell
(ray-dev) PS C:\Users\czgdp1807\Anyscale\ray\python> ray start --head
Enable usage stats collection? This prompt will auto-proceed in 10 seconds to avoid blocking cluster startup. Confirm [Y/n]:
Usage stats collection is enabled. To disable this, add `--disable-usage-stats` to the command that starts the cluster, or run the following command: `ray disable-usage-stats` before starting the cluster. See URL for more details.

Local node IP: 127.0.0.1
2025-06-05 16:10:27,199 ERROR services.py:1355 -- Failed to start the dashboard , return code 3221226505
2025-06-05 16:10:27,201 ERROR services.py:1380 -- Error should be written to 'dashboard.log' or 'dashboard.err'. We are printing the last 20 lines for you. See ' URL to find where the log file is.
2025-06-05 16:10:27,219 ERROR services.py:1424 --
The last 20 lines of C:\Users\CZGDP1~1\AppData\Local\Temp\ray\session_2025-06-05_16-10-17_593443_6000\logs\dashboard.log (it contains the error message from the dashboard):
Traceback (most recent call last):
 File "c:\users\czgdp1807\anyscale\ray\python\ray\dashboard\dashboard.py", line 246, in
 logging_utils.redirect_stdout_stderr_if_needed(
 File "c:\users\czgdp1807\anyscale\ray\python\ray\_private\logging_utils.py", line 49, in redirect_stdout_stderr_if_needed
 sys.stderr = open_log(stderr_fileno, unbuffered=True, closefd=False)
 File "c:\users\czgdp1807\anyscale\ray\python\ray\_private\utils.py", line 444, in open_log
 stream = open(path, **kwargs)
OSError: [WinError 6] The handle is invalid

--------------------
Ray runtime started.
--------------------

Next steps

 To connect to this Ray cluster:
 import ray
 ray.init()

 To terminate the Ray runtime, run
 ray stop

 To view the status of the cluster, use
 ray status

Let me see what's going on.

We are actively working on this together with @mattip. Will give an update in a day or two. In the meantime, are you able to use WSL?

@kolcker : For me it is now working with ray release 2.47. Kindly try again with latest version published on Friday.

I am interested in what was now the root cause and the fix, though.

@PhilippWillms thank you for information.

I still have the same issue after the clean installation of ray 2.47.
Installation `pip install -U "ray[train,tune]"` to the new Virtualenv

Script to reproduce

import ray
ray.init(include_dashboard=False)

the same error

E:\projects\ray_test\.venv15\Scripts\python.exe E:\projects\ray_test\test3.py
2025-06-15 18:07:39,219 ERROR services.py:1355 -- Failed to start the dashboard , return code 3221226505
2025-06-15 18:07:39,219 ERROR services.py:1380 -- Error should be written to 'dashboard.log' or 'dashboard.err'. We are printing the last 20 lines for you. See ' URL to find where the log file is.
2025-06-15 18:07:39,224 ERROR services.py:1424 --
The last 20 lines of C:\WINDOWS\TEMP\ray\session_2025-06-15_18-07-34_072267_35384\logs\dashboard.log (it contains the error message from the dashboard):
Traceback (most recent call last):
 File "E:\projects\ray_test\.venv15\Lib\site-packages\ray\dashboard\dashboard.py", line 246, in
 logging_utils.redirect_stdout_stderr_if_needed(
 File "E:\projects\ray_test\.venv15\Lib\site-packages\ray\_private\logging_utils.py", line 49, in redirect_stdout_stderr_if_needed
 sys.stderr = open_log(stderr_fileno, unbuffered=True, closefd=False)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "E:\projects\ray_test\.venv15\Lib\site-packages\ray\_private\utils.py", line 444, in open_log
 stream = open(path, **kwargs)
 ^^^^^^^^^^^^^^^^^^^^
OSError: [WinError 6] The handle is invalid

2025-06-15 18:07:39,414 INFO worker.py:1917 -- Started a local Ray instance.
[2025-06-15 18:07:42,064 E 35384 36528] core_worker.cc:513: Failed to register worker to Raylet: IOError: [RayletClient] Unable to register worker with raylet. Unknown error worker_id=01000000ffffffffffffffffffffffffffffffffffffffffffffffff

requirements.txt

annotated-types==0.7.0
attrs==25.3.0
certifi==2025.6.15
charset-normalizer==3.4.2
click==8.2.1
colorama==0.4.6
filelock==3.18.0
fsspec==2025.5.1
idna==3.10
jsonschema==4.24.0
jsonschema-specifications==2025.4.1
msgpack==1.1.1
numpy==2.3.0
packaging==25.0
pandas==2.3.0
protobuf==6.31.1
pyarrow==20.0.0
pydantic==2.11.7
pydantic_core==2.33.2
python-dateutil==2.9.0.post0
pytz==2025.2
PyYAML==6.0.2
ray==2.47.0
referencing==0.36.2
requests==2.32.4
rpds-py==0.25.1
six==1.17.0
tensorboardX==2.6.4
typing-inspection==0.4.1
typing_extensions==4.14.0
tzdata==2025.2
urllib3==2.4.0

Ray team, please let us know if you have any updates regarding status of this issue.

Thanks in advance.

Good point. Checking the details, I see that now my Windows has build ID **26100.4351**. Updates which were installed last Friday, 2025-06-11, are KB5061935 and KB5063060.

I'm running Windows 11 Build 22631.5472 and Ray 2.47.1 and still get the issue: **_OSError: [WinError 6] The handle is invalid_**
Any recommendation on what to do to resolve the issue. It was working ok on Windows 10 OS, but lately corporate made us upgrading to Windows 11.
Right now, i'm stuck because of this. Any help would be greatly appreciated. Thanks

Hi @kolcker 2.48 is cut and will be released within a few days.

Pleasee help with this error, after installing version 2.48 on windows 11 i get this after training:
`config = (
 SACConfig()
 .environment(
 TradingEnv, # NOT TradingEnv() or a wrapped function unless using register_env
 env_config={
 "train": train,
 "close": close,
 "close2": close2,
 "highs": highs,
 "lows": lows,
 "starting_index": 0,
 }
 )
 # .callbacks(EpisodeReturn)
 .training(
 gamma=0.9,
 actor_lr=0.001,
 critic_lr=0.002,
 train_batch_size_per_learner=32,
 )
 .env_runners(
 rollout_fragment_length=1000, # Collect more steps
 # num_env_runners=1,
 )
)
config = config.framework("torch")
for i in range(10):
 algo.train()
env = TradingEnv({
 "train": train,
 "close": close,
 "close2": close2,
 "highs": highs,
 "lows": lows,
 "starting_index": 10,
 })
policy = algo.get_policy()
total_reward = 0
done = False
state, _ = env.reset()
while not done:
 state, reward, done, _ = loaded_policy.compute_single_action(state)
 total_reward += reward`

Results in

`----> 9 policy = algo.get_policy()
 10 total_reward = 0
 11 done = False

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\ray\rllib\algorithms\algorithm.py:2398, in Algorithm.get_policy(self, policy_id)
 2391 @OldAPIStack
 2392 def get_policy(self, policy_id: PolicyID = DEFAULT_POLICY_ID) -> Policy:
 2393 """Return policy for the specified id, or None.
 2394
 2395 Args:
 2396 policy_id: ID of the policy to return.
 2397 """
-> 2398 return self.env_runner.get_policy(policy_id)

AttributeError: 'SingleAgentEnvRunner' object has no attribute 'get_policy'
`

@Hydraxx10 could you file a separate issue for the latest error since it's different from the issue this ticket is about. cc @sven1977 i think it's rllib issue now.


Question: July 2022 LWG issues?

I will add a note to the OP to make sure that `zip` implementation doesn't miss the changes from P2165R4. There's enough to P2165R4 that I think of it as more a standalone feature than simply a patch on `zip`, so I'll leave the separate feature issue.

July 2022 LWG issues
(Previous meta-issue: #2527)

At the July 2022 virtual plenary meeting, the resolutions of the following LWG issues were merged into the C++ Working Paper.

# ❔ Not yet analyzed

# ❌ Not applicable

If an issue requires no action from implementers, we mark it as N/A. Categories:
* Pure wording clarifications with nothing to implement (these can be changes to non-normative text like examples and informative notes, or wording cleanups to normative text that don't impact observable behavior)
 * [x] LWG-3659 Consider `ATOMIC_FLAG_INIT` undeprecation
 * MSVC STL has never deprecated this macro, and it might be impossible to reflect such deprecation in a library implementation
* Something that increases the restrictions placed on users, but implementers aren't expected to enforce those restrictions.
* Fixes for obviously broken wording, where implementers would have done the right thing anyways
 * [x] LWG-3709 LWG-3703 was underly ambitious
 * [x] LWG-3713 Sorted with respect to comparator (only)

# 😸 Already implemented
Sometimes we cite LWG issues in product code comments as we're implementing their proposed resolutions. When the resolutions are officially accepted, we should remove the citations (as the default assumption is that we're implementing what the Standard says). If something is especially subtle, we can convert the citation to mention the relevant Standard section.

Sometimes we should add test coverage - e.g. when the Standard begins requiring something that we were already doing, but weren't explicitly testing for.

* Already implemented, comments need to be removed and messages need to cite the Standard
 * [x] LWG-3670 *Cpp17InputIterator*s don't have integer-class difference types
 * [x] LWG-3705 Hashability shouldn't depend on `basic_string`'s allocator
* Implemented without comments
 * [x] LWG-3687`expected ` move constructor should move
 * [x] LWG-3703 Missing requirements for `expected requires is_void `
 * [x] LWG-3704 LWG-2059 added overloads that might be ill-formed for sets
 * [x] LWG-3708 `take_while_view::`_`sentinel`_'s conversion constructor should move

# 🩹 Patches an unimplemented feature
We should record this LWG issue in the GitHub issue tracking the feature. That way, we'll remember to verify it, but it doesn't represent net new work.
 * [x] LWG-3692 `zip_view::`_`iterator`_'s `operator ` is overconstrained
 * [x] LWG-3702 Should `zip_transform_view::`_`iterator`_ remove `operator ::value_type` and `iterator_category` should use `const F&`
 * [x] LWG-3617 `function`/`packaged_task` deduction guides and deducing `this`
 * Note that WG21-P1169 (static `operator()`) also changes these deduction guides.
 * [x] LWG-3701 Make `formatter , charT>` requirement explicit
 * PR out for review
 * [ ] LWG-3656 Inconsistent bit operations returning a count (implemented in #2880 )
 * [ ] LWG-3671 `atomic_fetch_xor` missing from `stdatomic.h` (implemented in #2905 )
 * [ ] LWG-3672 `common_iterator::operator->()` should return by value (implemented in #2907 )
 * [ ] LWG-3683 `operator==` for `polymorphic_allocator` cannot deduce template argument in common cases (implemented in #2879 )
 * [ ] LWG-3707 `chunk_view::`_`outer-iterator`_`::value_type::size` should return unsigned type (implemented in #2883 )
 * [ ] LWG-3710 The `end` of `chunk_view` for input ranges can be `const` (implemented in #2878 )
 * [ ] LWG-3711 Missing preconditions for `slide_view` constructor (implemented in #2946)
 * [ ] LWG-3712 `chunk_view` and `slide_view` should not be `default_initializable` (implemented in #2943 )
 * [ ] LWG-3715 `view_interface::empty` is overconstrained (implemented in #2946 )
 * [ ] LWG-3719 Directory iterators should be usable with default sentinel (implemented in #2902 )
 * [ ] LWG-3721 Allow an _arg-id_ with a value of zero for width in _std-format-spec_ (implemented in #2906 )
 * [ ] LWG-3724 _`decay-copy`_ should be constrained (implemented in #2903 )

@StephanTLavavej @CaseyCarter feel free to edit this if my analyzation for any LWG issue is incorrect, or P/R for any issue hasn't been voted into the working paper.

Wow, thanks @frederick-vs-ja! This is a ton of work. I'm going to tag this (and any relevant PRs) as "blocked" until we actually get approval for the proposed resolutions in the WG21 virtual plenary on 2022-07-25, but folks should feel free to go ahead and submit/review such PRs.

I believe every issue has now been processed. We have PRs out for the majority, I've recorded the two that patch `zip` in #2252, filed issues for the three that don't yet have pull requests, and submitted a PR to cleanup comments for the two we implemented speculatively with comments.

Thanks to everyone, and again extra-special thanks to @frederick-vs-ja for filing this issue and a majority of the resolution PRs.

P2321R2 `zip`
P2321R2 `zip`

Feature-test macro:

cpp
#define __cpp_lib_ranges_zip 202110L

Be sure to implement the resolutions of these LWG issues which patch this feature:

* LWG-3692 `zip_view::`_`iterator`_'s `operator ` is overconstrained
* LWG-3702 Should `zip_transform_view::`_`iterator`_ remove `operator<`?
* LWG-3773 `views::zip_transform` still requires `F` to be `copy_constructible` when empty pack
* LWG-3798 Rvalue reference and `iterator_category`
 + Affects `views::adjacent_transform`, see [\[range.adjacent.transform.iterator\]/1.1]( URL
* LWG-3848 `adjacent_view`, `adjacent_transform_view`, and `slide_view` missing `base` accessor

and the pertinent changes from P2165R4 and P2494R2.

Not that I would want to make promises, that I will create a MR any time soon. But I at least implemented some of the minor changes as well as zip_view in URL
I could use some help with:
- 5.1 & 5.2: Constructors and assignment operators for `std::tuple` and `std::pair`
- 5.4: Not sure which changes this requires
- 5.6: Is implemented as described in the paper. But are there functions whose noexceptness should be strengthened?

Hey @miscco , I assume this question is addressed to me? If so, thanks for asking! Unfortunately, currently I wont find the time to work on this. The missing parts in my branch are noexcept-strengthening, unit tests as well as `adjacent` and `adjacent_transform`.

Note that you do not need to implement everything in a go.

Could you do a squash and rebase on main? I would be happy to give it another round of review and Tam implemented most prerequisites

P2165R4 also changes `zip_view` etc. (making them no longer use `pair`) and `basic_common_reference` and `common_type` partial specializations.


Question: Getting an ERR_UNSUPPORTED_ESM_URL_SCHEME error?

Absolute paths (either unix or windows style) aren't valid URLs, though.

sh-session
$ node -p 'new URL("/Users/user/file.js").href'
internal/url.js:243
 throw new ERR_INVALID_URL(input);
 ^

TypeError [ERR_INVALID_URL]: Invalid URL: /Users/user/file.js

(same behavior in Chrome)

I 100% agree with passing the path through `url.pathToFileURL` first (that's the fixed I've applied in Jest), but I think it should be required on _all_ platforms, not just Windows.

Getting an ERR_UNSUPPORTED_ESM_URL_SCHEME error
### Describe the bug

I'm currently following the guide and at chapter 1 where I am now trying to run:

`npm run start` but getting the following error:

Error: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'f:'

I'm currently using `node 20.10.0`

Here's my `server.ts`

import { MikroORM } from "@mikro-orm/mysql";
import express from "express";
const app = express();
const port = 3000;

// initialize the ORM, loading the config file dynamically
const orm = await MikroORM.init();
console.log(orm.em); // access EntityManager via `em` property
console.log(orm.schema); // access SchemaGeneartor via `schema` property

app.get("/", (req, res) => {
 res.send("Hello World!");
});

app.listen(port, () => {
 return console.log(`Express is listening at URL
});

and my `package.json`

{
 "name": "server",
 "version": "1.0.0",
 "description": "",
 "scripts": {
 "build": "tsc && npx mikro-orm-esm cache:generate --combined",
 "start": "node --no-warnings=ExperimentalWarning --loader ts-node/esm src/server.ts",
 "test": "vitest"
 },
 "keywords": [],
 "author": "",
 "license": "ISC",
 "type": "module",
 "dependencies": {
 "@mikro-orm/core": "^6.4.15",
 "@mikro-orm/mysql": "^6.4.15",
 "express": "^5.1.0",
 "un": "^0.0.0",
 "zod": "^3.25.20"
 },
 "devDependencies": {
 "@eslint/js": "^9.27.0",
 "@mikro-orm/cli": "^6.4.15",
 "@mikro-orm/reflection": "^6.4.15",
 "@types/express": "^5.0.2",
 "@types/node": "^22.15.19",
 "eslint": "^9.27.0",
 "globals": "^16.1.0",
 "ts-node": "^10.9.2",
 "typescript": "^5.8.3",
 "typescript-eslint": "^8.32.1",
 "vitest": "^3.1.4"
 }
}

### Reproduction

Follow the guide and after setting up a basic entity, run `npm run start`

### What driver are you using?

MySQL (mikro-oorm/mysql 6.4.15)

### MikroORM version

6.4.15

### Node.js version

20.10.0

### Operating system

Windows 11

### Validations

- [x] Read the Contributing Guidelines.
- [x] Read the docs.
- [x] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
- [x] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord.
- [x] The provided reproduction is a minimal reproducible example of the bug.

We encountered the same issue on Windows. It's a problem that occurs when using dynamic imports with absolute paths. On Windows, the drive letter in the path (e.g., `f:`) is interpreted as a protocol, since `import()` accepts URLs. As a result, it requires the `file:` protocol, which can be properly constructed using `pathToFileURL`.

@the-black-viper You can resolve this by adding a `dynamicImportProvider` to the init config:

typescript
import { MikroORM } from '@mikro-orm/mysql';
import { pathToFileURL } from 'node:url';

const orm = await MikroORM.init({
 dynamicImportProvider: (path) => import(pathToFileURL(path).href),
});

@B4nan This can be fixed by updating the defaults here:

 URL

 URL

Both should use pathToFileURL as shown above.

For reference, here's the related Node.js issue: URL

Thanks, I was expecting something like that. Could you send a PR please? I would rather not try to fix this in the blind.

@Ruby184 Adding `dynamicImportProvider` to the config along with explicitly initializing the config as follows fixed the issue:

// mikro-orm.config.ts
const config: Options = {
 ...
 debug: true,
 dynamicImportProvider: (path) => import(pathToFileURL(path).href),
};

// server.ts

const orm = await MikroORM.init(config);

ES Module loading with abolute path fails on windows
* **Version**: v13.8.0
* **Platform**: Windows 10 64-bit version 1809
* **Subsystem**: "esm" maybe?

### What steps will reproduce the bug?

1. Create a new project in `C:\prog\node\genast`
2. Add "type": "module" in package.json
3. Make a "other.js" with some content, like

const thing = "hej"
export default thing

4. Make "index.js" with the following:

import other from "./other.js"
console.log(other)

5.

hej

(as expected!)

6. Change index.js to:

import other from "C:\\prog\\node\\genast\\other.js"
console.log(other)

It errors out with:
internal/modules/run_main.js:54
 internalBinding('errors').triggerUncaughtException(
 ^

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only file and data URLs are supported by the default ESM loader
 at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:33:11)
 at Loader.resolve (internal/modules/esm/loader.js:85:40)
 at Loader.getModuleJob (internal/modules/esm/loader.js:188:40)
 at ModuleWrap. (internal/modules/esm/module_job.js:42:40)
 at link (internal/modules/esm/module_job.js:41:36) {
 code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}

Using forward- och backward slashes doesn't make a difference.

### How often does it reproduce? Is there a required condition?

100%

### What is the expected behavior?

Absolute includes should work the same as relative

### What do you see instead?

### Additional information

The same issue is seen with both static ES imports and dynamic ES imports.

Sorry about the sources being one-linered. I can't figure out how to convince the markdown here how to allow multiline code blocks.

You'd need to write `file:///c:/x/y/z` instead of `c:/x/y/z`

**As per node.js documentation:**

The specifier of an import statement is the string after the from keyword, e.g. 'path' in import { sep } from 'path'. Specifiers are also used in export from statements, and as the argument to an import() expression.

There are four types of specifiers:

- Bare specifiers like 'some-package'. They refer to an entry point of a package by the package name.

- Deep import specifiers like 'some-package/lib/shuffle.mjs'. They refer to a path within a package prefixed by the package name.

- Relative specifiers like './startup.js' or '../config.mjs'. They refer to a path relative to the location of the importing file.

- Absolute specifiers like 'file:///opt/nodejs/config.js'. They refer directly and explicitly to a full path.

Bare specifiers, and the bare specifier portion of deep import specifiers, are strings; but everything else in a specifier is a URL.

Only file: and data: URLs are supported. A specifier like ' URL may be supported by browsers but it is not supported in Node.js.

Specifiers may not begin with / or //. These are reserved for potential future use. The root of the current volume may be referenced via file:///.

@kiranpwr1260 thanks!

It works on mac os though. Maybe not the same way you posted, but it works according to expectation.

On macos, if you do this:

1. type: module in package.json
2. In index.js, put

import "/Users/ /prog/node/genast/other.mjs"

3. In other.mjs, put `console.log('wep')`
4. `node index.js`
Outputs, as expected, `wep`

I expected the same to be true on windows, hence the issue report :)

Yeah, I think this is either a bug on macos & linux or windows.

Mac and Linux (alpine docker image, to be precise):

sh-session
$ echo 'export default "hello!";' > file.mjs
$ node -e 'import(require.resolve("./file.mjs")).then(console.log, console.error)'
(node:28) ExperimentalWarning: The ESM module loader is experimental.
[Module] { default: 'hello!' }

Windows:

sh-session
$ echo 'export default "hello!";' > file.mjs
$ node -e 'import(require.resolve("./file.mjs")).then(console.log, console.error)'
(node:3036) ExperimentalWarning: The ESM module loader is experimental.
Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only file and data URLs are supported by the default ESM loader
 at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:33:11)
 at Loader.resolve (internal/modules/esm/loader.js:85:40)
 at Loader.getModuleJob (internal/modules/esm/loader.js:188:40)
 at Loader.import (internal/modules/esm/loader.js:163:28)
 at importModuleDynamically (internal/modules/cjs/loader.js:1094:27)
 at exports.importModuleDynamicallyCallback (internal/process/esm_loader.js:37:14)
 at Object. (C:\Users\IEUser\script.js:1:16)
 at Module._compile (internal/modules/cjs/loader.js:1151:30)
 at Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
 at Module.load (internal/modules/cjs/loader.js:1000:32) {
 code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}

With `url.pathToFileURL` it works, so we can use that in Jest. I'd still consider this a bug in node, though

the issue is that `C:/` is ambiguous with a URL. (like `HTTP:/`) and we use URLs in the loader.

From a user perspective it's really weird that absolute paths work for some OSes, but not all. Would it make sense to throw on absolute paths without `file://` on mac and linux as well?

Yeah, I've changed to use that in Jest now, but it's still odd to me that it behaves differently across OSes, even though `pathToFileURL` fixes it. You have to actually test on windows to notice absolute paths doesn't work (in our case, we only tested windows on Node 12, while our test only runs on Node 13), then find about that API (followed by either fixing your own code, or reporting it to some library).

Maybe the error message could include something about passing the file path provided through `pathToFileURL`? Should hopefully mean less googling for people hitting it

This is because `C:/path/to/module.js` gets parsed as:

js
URL {
 scheme: "c:",
 host: null,
 path: "/path/to/module.js",
}

And **Node** doesn’t know how to deal with a URL with a scheme of `c:`.

What you want is `file:///C:/path/to/module.js`:

js
URL {
 scheme: "file:",
 host: "",
 path: "/C:/path/to/module.js",
}

Yeah, I understand _why_ it happens, I just disagree with the behavior that absolute paths are treated differently between platforms, and would prefer it to just accept either relative paths or absolute paths that has the `file` protocol to avoid the foot gun of code working fine on one platform and not the other.

That ship might have sailed though 🙂

@SimenB it doesn't accept paths, it accept urls. like i said, the correct thing to do if you have a path is to use pathToFileURL.

So to summarize and clarify, ESM importing right now works like this:

- relative paths:
 - works on mac/linux
 - works on windows
- absolute paths:
 - works on mac/linux
 - **doesn't work on windows**
- "local file urls"
 - works on mac/linux
 - works on windows

For my money, I would love to have absolute paths work on windows as well. It only makes sense that a function to include other files can actually handle paths. At least when running in a node.js environment.

I understand the difficulties differentiating between a windows absolute path and an url, since the drive letter could be confused with a URL scheme. And single-letter schemes seem to be valid according to this RFC: URL . But I think it's safe to assume that introducing single-letter URL schemes in a world where windows exists is not likely to happen

@SimenB they aren't valid *absolute* urls, but they are valid as relative urls ( URL which is why they're accepted in `import`.

I'm not sure how we could detect the intent of someone when they typed it, since that's all that separates an absolute path from a relative path absolute url string.

Aha, gotcha! Didn't think about the case when it's a `path` with a separately provided `base`. Thanks for being patient with me 👍

sh-session
$ node -p 'new URL("/Users/user/file.js", " URL
 URL

So what essentially happens is `import(specifier) => import(new URL(specifier, import.meta.url))` to resolve the absolute path? In that case I don't really have any ideas on how to best detect it 😅 I assume being stricter (i.e. only treating paths starting with `./` as relative and not `/`) is not an option? Would probably be bad for browser interop.

I think this should be highlighted in the docs somewhere, although I'm not entirely sure _where_. `pathToFileURL` is only mentioned in one place in the current ESM docs, and that's in an example transform loader. Maybe in the section talking about relative specifiers to `import()`? The 2 relative examples use `./`, might be an idea to mention `/` will be treated as relative to the root of the FS, which breaks on Windows? While I now (finally) understand why it behaves the way it does, it feels like a footgun if you're migrating from `require` to `import`.

As discussed, this behaviour is by design. A docs PR to help clarify further would be very welcome.

You can work with absolute paths in windows using simport for dynamic imports.
It also makes you feel like you using promisified `require` with `ESM` support.

js
// other.mjs
export const hello = 'world';

js
// index.mjs
import {createSimport} = require('simport');
const simport = createSimport(import.meta.url);

const {hello} = await simport('c:\\other.mjs');
console.log(hello)
// outputs
'world';

Can you please provide the line number of the source code that causes the error, in the error message?

The PR stalled, so the docs did not get updated to actually explain what is happening.
Please re-open the issue and/or fix up and merge the PR!

Specifically, this PR was supposed to address how to dynamically import absolute paths in the docs.

Even more specifically, this proposed change (rendered version here) is actually the solution (I tested it, and it works!):

js
import { pathToFileURL } from 'url';

const fileUrl = pathToFileURL('C:\\path\\to\\file.js').href;

await import(fileUrl)

It would be nice to be able to load an absolute path... in this case I don't have a way to set the current working directory, and `file:///c:/...` or `file://c:/...` do not work. It ends up prepending the current path (which again I don't have control over) and attempting to load like `c:\tmp\file:\c:\` for `node file:///c:/tmp/shell/node_modules/...` I just happen to be in c:\tmp when testing this, but that is not likely to be the path.


Question: Drag-and-drop doesn't work under chromium (wayland)?

hm, odd. it surely works for me. Anything else you can find to trigger this? Some config opt?

Drag-and-drop doesn't work under chromium (wayland)
### Regression?

Yes

### System Info and Version

 System/Version info

sh
Hyprland, built from branch main at commit 688fe5c14781c63a1db23d4d02bf239283068ff6 dirty (windowrules: add fullscreenstate field (7466)).
Date: Sat Aug 24 03:42:14 2024
Tag: v0.42.0-62-g688fe5c1, commits: 5131

flags: (if any)

System Information:
System name: Linux
Node name: Arch
Release: 6.10.5-2-cachyos-lto
Version: #1 SMP PREEMPT_DYNAMIC Thu, 15 Aug 2024 08:14:57 +0000

GPU information:
64:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Phoenix1 [1002:15bf] (rev c7) (prog-if 00 [VGA controller])

os-release: NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL=" URL
DOCUMENTATION_URL=" URL
SUPPORT_URL=" URL
BUG_REPORT_URL=" URL
PRIVACY_POLICY_URL=" URL
LOGO=archlinux-logo

plugins:

### Description

I tried bisect and went back to aquamarine before drag-and-drop didn't work either, probably influenced by other factors.

### How to reproduce

1. with `chromium --ozone-platform=wayland` command startup chromium
2. Click on a link tag and drag

### Crash reports, logs, images, videos

 URL

I didn't describe it well, it's possible to drag a link into a new tab, followed by no response to left mouse click anywhere in chromium.

Happening for me as well on brave since yesterday. After dragging something on the page or even dragging toolbar elements like extensions, the browser gets stucked with a hand symbol in place of the cursor.

See this:
- URL
- URL

drag-and-drop loop when dealing with misbehaving/buggy or malicious
Wayland compositors.

as described in the linked crbugs.

Happening for me as well on brave. After dragging something on the page or even dragging toolbar elements like extensions, the browser gets stucked with a hand symbol in place of the cursor.

See this:
- URL
- URL

drag-and-drop loop when dealing with misbehaving/buggy or malicious
Wayland compositors.

as described in the linked crbugs.

Similar issue: URL

This issue started appearing again on the newer versions (nightly/unstable) of the browsers.

I have this issue with Brave and Google Chrome, but no with Chromium.

Brave: Version 1.69.153 Chromium: 128.0.6613.85 (Official Build) (64-bit)
Google Chrome: Version 128.0.6613.84 (Official Build) (64-bit)
Chromium: Version 127.0.6533.99 (Official Build) Fedora Project (64-bit)

 System/Version info

sh
Hyprland, built from branch at commit 918d8340afd652b011b937d29d5eea0be08467f5 (flake.lock: update).
Date: Tue Jun 25 12:06:02 2024
Tag: v0.41.2, commits: 4886

flags: (if any)

System Information:
System name: Linux
Node name: fedora
Release: 6.10.6-100.fc39.x86_64
Version: #1 SMP PREEMPT_DYNAMIC Mon Aug 19 14:35:32 UTC 2024

GPU information:
00:02.0 VGA compatible controller [0300]: Intel Corporation Skylake GT2 [HD Graphics 520] [8086:1916] (rev 07) (prog-if 00 [VGA controller])

os-release: NAME="Fedora Linux"
VERSION="39 (Workstation Edition)"
ID=fedora
VERSION_ID=39
VERSION_CODENAME=""
PLATFORM_ID="platform:f39"
PRETTY_NAME="Fedora Linux 39 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:39"
DEFAULT_HOSTNAME="fedora"
HOME_URL=" URL
DOCUMENTATION_URL=" URL
SUPPORT_URL=" URL
BUG_REPORT_URL=" URL
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=39
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=39
SUPPORT_END=2024-11-12
VARIANT="Workstation Edition"
VARIANT_ID=workstation

@hdm9527 I know, but for some reason Brave and Google Chrome have the issue and Chromium doesn't.

@gmr458 I have the same issue in the most updated Chromium 130.0.6681.0

Same problem here, with Brave and Chromium

Hyprland, built from branch at commit 9a09eac79b85c846e3a865a9078a3f8ff65a9259 (props: bump version to 0.42.0).
Date: Wed Aug 7 19:17:10 2024
Tag: v0.42.0, commits: 5069

flags: (if any)

Got the same issue on Brave. I think it started after nvidia 560 drivers.

 System/Version info

sh
Hyprland, built from branch at commit 9a09eac79b85c846e3a865a9078a3f8ff65a9259 (props: bump version to 0.42.0).
Date: Wed Aug 7 19:17:10 2024
Tag: v0.42.0, commits: 5069

flags: (if any)

System Information:
System name: Linux
Node name: karamanliev
Release: 6.10.6-arch1-1
Version: #1 SMP PREEMPT_DYNAMIC Mon, 19 Aug 2024 17:02:39 +0000

GPU information:
26:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU104 [GeForce RTX 2070 SUPER] [10de:1e84] (rev a1) (prog-if 00 [VGA controller])
NVRM version: NVIDIA UNIX x86_64 Kernel Module 560.35.03 Fri Aug 16 21:39:15 UTC 2024

os-release: NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL=" URL
DOCUMENTATION_URL=" URL
SUPPORT_URL=" URL
BUG_REPORT_URL=" URL
PRIVACY_POLICY_URL=" URL
LOGO=archlinux-logo

The Plasma/KWin people had a similar problem. Perhaps related?
 URL

From my testing, it seems likely that a regression occurred on Chromium side between version 127 and 128 and @gmr458 message seems to point to the same direction.

URL

Mhm, so looks like a bug in chromium that could be somewhat alleviated on our side

Just switched to `hyprland-git` on Arch and I can confirm issue is solved with the fix :+1:

yes because it's a chromium regression that can be alleviated if hyprland better responds to invalid protocol usage

Click and drag cursor cannot release on Chromium(Wayland mode)
### Bug or Regression?

Bug

### System Info and Version

 System/Version info

sh
Hyprland, built from branch main at commit 4842eb83b444418ad3fe1901d645dd02224989e5 (helpers: make shm_open() portable after 8bcccf9f0f0f (6471)).
Date: Thu Jun 13 20:20:14 2024
Tag: v0.41.1-2-g4842eb83, commits: 4820

flags: (if any)

System Information:
System name: Linux
Node name: Arch
Release: 6.9.3-2-cachyos-echo-lto
Version: #1 SMP PREEMPT_DYNAMIC Thu, 06 Jun 2024 06:05:56 +0000

GPU information:
64:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Phoenix1 [1002:15bf] (rev c7) (prog-if 00 [VGA controller])

os-release: NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL=" URL
DOCUMENTATION_URL=" URL
SUPPORT_URL=" URL
BUG_REPORT_URL=" URL
PRIVACY_POLICY_URL=" URL
LOGO=archlinux-logo

plugins:

### Description

Clicking on a tab or button and dragging it under chromium does not release the cursor.

### How to reproduce

1. start chromium with `chromium --ozone-platform=wayland`.
2. clicking on a tab or button on a web page and dragging it.

### Crash reports, logs, images, videos

 URL

this is a broken chromium impl it seems, at least for the home screen icon drag, as that is broken on sway too. The getting stuck is a hyprland bug.

it surely was released on my end. Make sure you're on the right version with `hyprctl version`

Hyprland, built from branch main at commit 91fe58f8f278d126852877eadc87c50ca7b9b78d (window: improve swallowing functionality).
Date: Sun Jun 16 00:20:41 2024
Tag: v0.41.1-18-g91fe58f8, commits: 4836

flags: (if any)

I commented out the chromium-flags.conf configuration and just started chromium with the chromium --ozone-platform=wayland command and it still doesn't release the cursor

The cursor is not released even if the text is selected and dragged.

it... works on my end but is really wonky, I'll try to look at it tomorrow

It seems to be worse, before it was the current tab not releasing the cursor.

 URL

Helvum didn’t get fixed for me with the previous patch. Trying your latest one now that I’ve got a good dev env set up.

@vaxerski It still does not work in Helvum for me so I’ll open a new issue

For the people that are not using the `-git` package version, when are you going to release a new version? :)

Yes, it seems it's happening again for me as well

Hyprland, built from branch at commit 9a09eac79b85c846e3a865a9078a3f8ff65a9259 (props: bump version to 0.42.0).
Date: Wed Aug 7 19:17:10 2024
Tag: v0.42.0, commits: 5069

flags: (if any)

Happening to me as well. When setting the ozone platform to X11 it goes away but you get those big gaps around the modal windows.

Hyprland, built from branch at commit 9a09eac79b85c846e3a865a9078a3f8ff65a9259 (props: bump version to 0.42.0).
Date: Wed Aug 7 19:17:10 2024
Tag: v0.42.0, commits: 5069

flags: (if any)

Yea, seems like this is happening to me too

$hyprctl version
Hyprland, built from branch at commit 9a09eac79b85c846e3a865a9078a3f8ff65a9259 (props: bump version to 0.42.0).
Date: Wed Aug 7 19:17:10 2024
Tag: v0.42.0, commits: 5069

flags: (if any)

Uploading 2024-09-04 19-17-10.mp4…

IT IS FIXED IN GIT PLEASE STOP POSTING THAT IT HAPPENS IN 0.42


Question: Vue + Markdown example not working?

I finally found some time to fix it all and also finished upgrading to Vite 3 :) Everything should be working in latest, feel free to give it a try and let me know how you go. Raise a new issue if there's any problems.

Vue + Markdown example not working
npm init vitebook vitebook

Select Vue & Markdown.

npm install
echo "# Hello World" > src/test.md
npm run vitebook:dev

The Markdown file will not show up.

I get the same issue for the svelte example too, i get an empty vitebook default theme page with no menu, it seems like it loads correctly, it redirects to the default example component, but it never shows it.

npm init vitebook my-vitebook-test-app -- --template svelte
cd my-vitebook-test-app
npm i
npm run vitebook:dev

I also get the same behaviour on an previously perfectly working vitebook page so i suspect some (peer?) dependency somewhere was bumped and broke it.

Worth noting is that running `npm run vitebook:build && npm run vitebook:preview` works fine and builds a working page.

After too many hours of attempts to debug why i got a blank screen i tried to downgrade some dependencies and found that it works fine by just "downgrading" to an explicit dependency on vite `2.8.4` but not with newer ones.

@possan I facing the same issue with react, you downgrade vite to 2.8.4?

@possan Might be related to #72, as both are very recent problems, and one might lead to the other, i suggest that we focus efforts there to address this issue, and close this one (and other similar issues) once it's solved

Broken since 0.18.6
HI this is a great idea so far

Im facing an issue where nothing works anymore after updating to 0.19.0 and beyond

I even tried the same changes for your documentation site and it shows that it wasnt working

This library has alot of potential to be better than storybook...

reproducible link: URL
if you update to 0.19.0 and after the pages wont show anymore

If you're talking about the missing source files error I got the same kind of issue, but with other versions:

git clone URL
cd svelte-babylon
# until here everthing works
npm i -d vite # at the time of writing going from vite 2.7.0 to vite 2.9.2
npm run vitebook:dev

Error message:
`Sourcemap for "/home/user/Projects/svelte-babylon/library/node_modules/@vitebook/client/dist/client/addons/events/EventsAddon.svelte" points to missing source files`

My vitebook versions are `^0.23.4`, so I assume, that the error originates from `vite`.

There have been similar reports:
- URL
- URL

@Myrmod hmm I still think the issue is caused by vitebook rather than vite. Since im able to run vite at `2.9.1` with vitebook `0.18.6`

Thanks for reporting this @GZLiew, i'm facing the same issue, tried downgrading vitebook to `0.18.6` as a temporary workaround, couldn't get it to work, but i think it's due to a completely unrelated reason:

4:50:50 AM [vite] Internal server error: /home/hbr/repos/github/itatiaia-portal-web/node_modules/@vitebook/client/dist/client/App.svelte:29:16 Cannot have an {:else} block outside an {#if ...} or {#each ...} block
 Plugin: vite-plugin-svelte
 File: /home/hbr/repos/github/itatiaia-portal-web/node_modules/@vitebook/client/dist/client/App.svelte
 27 | const file = "node_modules/@vitebook/client/dist/client/App.svelte";
 28 |
 29 | // (17:0) {:else}
 ^
 30 | function create_else_block(ctx) {
 31 | let switch_instance;

I'm also using the exact vite version of you (`2.9.1`)

I've confirmed that the issue is with Vite `2.9.1` to get things working I recommend anchoring your Vite version to `2.8.6` which is the last working version for Vitebook

@joseaquino nope still doesnt work for me can you share me a sample repo?

I tested it by bumping vite to `2.8.6` in their example stackblitz URL

@joseaquino you're right! 👍🏻 hard fixing the dependencies of vite to `2.8.6` does fixes the solution. Guys Im going to mark this as the solution until vite solves the problem for us.

The issue still persists for me with Vite@2.9.8.

Vite@2.8.6 working well, I'm running Vitebook with Vitest and testing my components by using the preact testing library.

It looks like vite@2.8.6 is the last working version

I was able to revive the examples (I tried `examples/svelte-kit`) with pinning vite-plugin-svelte and vite package versions in package.json:

jsonc
// package.json
{
 ...
 "devDependencies": {
+ "@sveltejs/vite-plugin-svelte": "1.0.0-next.36",
 ...
+ "vite": "2.8.6"
 },
+ "pnpm": {
+ "overrides" : {
+ "@sveltejs/vite-plugin-svelte": "1.0.0-next.36",
+ "vite": "2.8.6"
+ }
+ },
 ...
}


Question: Explain how to use `git pr list --template` feature, or at least give some hints and an example?

@heaths Looks pretty good! I think you're definitely on to something here.

Does `text/tabwriter` work when the lines passed in contain ANSI escape sequences? I would have guessed that it doesn't recognize these escape sequences as non-printable and that they can mess up horizontal alignment.

Explain how to use `git pr list --template` feature, or at least give some hints and an example
### Describe the feature or problem you’d like to solve

The `gh pr list` command has an argument `--template`; the only documentation for it that I could find states:

But what does that mean? For Go aficionados this might be crystal clear, but others (including me) may have no clue at all what that means.

### Proposed solution

Explain this more. At the very least, perhaps a link to an external site could be inserted that explains what a "Go template" is. A minimal example would also be very nice.

### Website
 URL

### GitHub cli/cli repo
Check the comments made by @heaths on ticket #3488

Running `gh help template` (or maybe "templates") will show you lots of information with examples. `gh pr list --help` should say that like I know other commands do. Does it not?

@LangLangBart thanks that's helpful, I didn't find it

@heaths neither `gh help template` nor `gh help templates` does anything useful for me. Aha, but combined with what @LangLangBart pointed out, I am guessing `gh help formatting` is what you had in mind?

As it is `gh pr list --help` does not mention this or anything like it. And e.g. `gh issue list --help` also lists a `--template` option, but again without any examples or pointers where to find out more.

If

 -t, --template string Format JSON output using a Go template

was changed to, say

 -t, --template string Format JSON output using a Go template, for more details consult `gh help formatting`

or if those help pages just would mention this at the bottom, that'd be swell

@fingolfin Thats seems like a good idea. Will you make the PR ?

___

Once you have selected your values for your `--json` flag and made them pretty with `--template`, you can then pass your command through fzf for an interactive experience.

Take advantage of the environment variable `GH_FORCE_TTY`, to enable terminal-style output even when the output is redirected (see `gh environment` for more details about it).
### Example:
#### Code

zsh
# .zshrc gh pr list command extended with fzf, see the man page (man fzf) for an explanation of the arguments.
function ghpr {
 [[ ! "$(git rev-parse --is-inside-work-tree)" ]] && return 1
 GH_COMMAND='gh pr list --state open --json number,author,additions,deletions,updatedAt,title --template "
 {{- tablerow (\"PR\" | color \"blue+b\") (\"LAST UPDATE\" | color \"blue+b\") (\"AUTHOR\" | color \"blue+b\") \"\" \"\" (\"TITLE\" | color \"blue+b\") -}}
 {{- range . -}}
 {{- tablerow (printf \"#%v\" .number | color \"green+h\") (timeago .updatedAt | color \"gray+h\") (.author.login | color \"cyan+h\") (printf \"+%v\" .additions | color \"green\") (printf \"-%v\" .deletions | color \"red\") .title -}}
 {{- end -}}" --search'
 FZF_DEFAULT_COMMAND="$GH_COMMAND ${1:-\"\"}" \
 GH_FORCE_TTY=100% fzf --ansi --disabled --no-multi --header-lines=1 \
 --header $'CTRL+B - Browser | CTRL+D - Toggle Diff | CTRL+X - Checkout\nCTRL+E - Edit | CTRL+I - Toggle Info | CTRL+Y - Comment' \
 --prompt 'Search Open PRs >' --preview-window hidden:wrap \
 --layout=reverse --info=inline --no-multi \
 --bind "change:reload:sleep 0.25; $GH_COMMAND {q} || true" \
 --bind 'ctrl-b:execute-silent(gh pr view {1} --web)' \
 --bind 'ctrl-d:toggle-preview+change-preview(gh pr diff {1} --color always)' \
 --bind 'ctrl-i:toggle-preview+change-preview(gh pr view {1} --comments)' \
 --bind 'ctrl-e:accept+execute(gh pr edit {1})' \
 --bind 'ctrl-x:accept+execute(gh pr checkout {1})' \
 --bind 'ctrl-y:accept+execute(gh pr comment {1})'
}

#### GIF
In the example I used `nano` as editor to write the comment, but you can set your own with `gh config set editor ...`.

Now that I'm back at a workstation, running `gh issue list --help` does show:

`gh pr list --help` shows the same thing:

Given I was on my phone with no easy way to look up, I misremembered "template" (or variant) instead of "formatting", but that's what I was trying to remember.

If the maintainers are good with adding text to the individual usage line, do it at the following locationand it'll apply to all commands using it: URL

I would be thrilled for better docs around this and am cool with heaths's suggestion. I would also be fine with further additions added to `gh help formatting` if there are ways to make that doc more helpful.

Provide Go template functions to format table output
### Describe the feature or problem you’d like to solve

Related to #3487 there was no great way to provide table output like you do for most (or perhaps all) your other commands like `gh issue list`. You expose a `color` function and I can use `printf` to align and pad strings, but what would be really great is a `table` function.

### Proposed solution

The Docker CLI lets you specify a `table` function that works like this:

bash
docker images --format "table {{.Repository}}\t{{.Tag}}"

It would then output something like:

none
REPOSITORY TAG
azuresecuritykeyvaultcertificatestests latest
dotnet-sdk 5.0-ubuntu-18.04
jupyter/base-notebook latest

This is done via, mostly, URL It is effectively like `{{range}}` but splits on tabs and determines optimal width automatically.

Alternatively, if there was some way to declare the scope of nodes to format as a table automatically, perhaps you could - when specified - format that with survey as you do your own table output.

Looking through the built-in formatting more, I wonder if this could instead be used for built-in command output rather than manually formatting the output. The crux of Docker's `table` template function is "text/tabwriter" which uses elastic tabstops, so even paginated results should align properly. I'm interested in taking a crack at this if there's interest in accepting it. At the minimum, I think making it available for `gh api --template` would be a good start, and whether to adopt it for built-in commands could be a separate discussion. You could combine that with your existing `color` functions, though it seems you'll need to add a couple more color options to match what you use currently for built-in command colored output.

@heaths I'd be interested in having a table template function!

If our templating support gets full-featured enough, we could theoretically switch built-in output to the template approach. That would also serve as a starting template for someone who would like to tweak the output in their own scripts. But for now, I'd propose expanding `--template` functionality first, then exploring if built-in commands can switch to it separately.

I am +1 on these ideas but defer to @mislav on the details as he is the one who's been thinking most about programmatic output like this.

@heaths You are welcome to send a PR adding a `table` helper. I'm interested in how it will work 👍

I'll try to take a look this weekend. I need to play around with things a little (probably just a scratch program) to see if `{{with pipeline}}` will work since your GraphQL results typically have the juicy bits nested lower. Using Docker's `table` as a pattern, it only wants leaf nodes and seems to expect strings. If `{{with}}` doesn't work, may need to tweak this `table` function a little more to handle nesting, but I don't foresee a problem.

After playing around with a few options last night, I think this might be better served by a PR to the text/template package itself. Docker's "function" is really just a hack: if the template starts with "table" (or a couple others like "raw"), that prefix is stripped from the template and it does some simple string transforms on the remaining pipeline. It works well when your output is already an array, but if you're array is arbitrarily nested (like a GraphQL query might often end up), it wouldn't work. Even if we scanned the string for any occurrence of "table" (or similar), there's no obvious end. The `text/template/parse.Tree` that is compiled is where the change really needs to happen. I can also make it work with 2 templates, but it seems like an even bigger hack and not very user-friendly to have two arguments for a table, like `--template '{{with .Nodes}}{{template "table"}}{{end}}` then, if you use a table, `--template-table '{{range .}}{{.Id}}{{"\t"}}{{.Name}}{{end}}'`.

For now, let me see if there's any interest in a change to the text/template package. I think it could be an all-around useful addition.

Thanks for exploring all this! I have little faith that the proposal to `text/template` will be accepted, since I think you will be told to just implement this programmatically (either via custom helper function or nested templates), but even if it doesn't get into Go, I think we can just invent some approach ourselves. For example:

gotemplate
{{range .}}{{ tableRow .number .title .author }}{{end}}
{{tableRender}}

I've been considering some similar approaches under the assumption it won't be accepted because it actually emits formatted text, which seems out of scope.

One idea was similar to what you suggested, but the problem is that you can't pass fields to functions if, for example, you wanted to color text. Maybe some was to do that would be more acceptable, like inline pipelines via syntax like (partial) `{{tableRow (.I'd | color "green")}}`.

If not even that, I've been thinking that we could pre-parse the template and, like Docker CLI, manipulate it to make another named template internally. I'll continue to play around with this in my scratch program.

Well...I'm shocked. Apparently parenthesis already work. While perusing through the text/template/parse code and making a few preliminary changes to support my proposal, I failed to see how parsed parenthesis were used.

This commit shows my idea above already works:

image

I'll proceed with this course of action, but would also like to add a couple more helper functions that I think will also help in the future if we wanted to replace the built-in formatting with this new set of functions, e.g. `truncate`, though I think I might reverse the `uint` and `string` params so you can do either `{{truncate 30 .Title | color "green"}}` or `{{.Title | truncate 30 | color "green"}}`.

After further consideration I propose support for multiple tables, which could be beneficial down the road. Effectively, this:

`{{table}}{{range .Books}}{{row (.Title | truncate 30 | green) .ISBN13}}{{end}}{{endTable}}`

We could use `{{tableRow}}` as well, but with `{{table}}` I figured it would be fairly obvious and slightly prefer brevity over specificity in a rather limited scope like this.

`{{table}}` would set the current table for subsequent `{{row}}` calls, while `{{endTable}}` would flush the current table an nilify it.

C-style escape sequences within the text won't be interpreted, but literal escapes will. While the web forms appear to prevent this (or at least make it harder), I was able to add an issue with literal escapes in both the title and text. Seems I'll have to create a replacer that perhaps JS-encodes all but `\ux1b` used for vterm sequences.

Actually, looks like this shouldn't be a problem:

bash
$ gh api graphql -f query='query {
 repository(name: "templ", owner: "heaths") {
 issue(number: 1) {
 number
 title
 bodyText
 }
 }
}'

json
{
 "data": {
 "repository": {
 "issue": {
 "number": 1,
 "title": "Title:\u0008 with escape",
 "bodyText": "This\tbody has\nescape sequences"
 }
 }
 }
}

Working on some tests and docs, but a preview of using a table template for `gh issues list`:

bash
bin/gh issue list --json number,title,labels,updatedAt --template '{{table}}{{range .}}{{row (.number | printf "#%v" | color "green") (.title | ellipsis 50) (.labels | pluck "name" | join ", " | printf "(%s)" | color "gray") (timeago .updatedAt | color "gray")}}{{end}}{{endTable}}'

image

For those wondering what gh template ended as. Here is an example:

gh issue list --milestone "GD 2.3.3" -s all --json number,title,url -t'{{range .}}{{tablerow .number .url .title}}{{end}}{{tablerender}}'

- "range ." loop over all elements of the collections.
- tablerow prepares the table using the element of each item of the collections (the names are the ones provided to --json option
- tablerender will render the final table
- any space between {{ }} will be used as is
- it is not possible to use anything outside {{}} as it will be interpreted as option for the cmd or error out (it would be nice to be able to pass a file btw)

Hope that helps.

PS: It would be awesome to add that to the doc here and there.

I did add that to the `formatting` help topic, though that page you reference doesn't seem to be rendering it correctly:

With `--template`, the provided Go template is rendered using the JSON data as input.
For the syntax of Go templates, see: URL

The following functions are available in templates:
- `autocolor`: like `color`, but only emits color to terminals
- `color `: colorize input using URL
- `join `: joins values in the list using a separator
- `pluck `: collects values of a field from all items in the input
- `tablerow ...`: aligns fields in output vertically as a table
- `tablerender`: renders fields added by tablerow in place
- `timeago `: renders a timestamp as relative to now
- `timefmt `: formats a timestamp using Go's Time.Format function
- `truncate `: ensures input fits within length

EXAMPLES
 # format issues as table
 $ gh issue list --json number,title --template \
 '{{range .}}{{tablerow (printf "#%v" .number | autocolor "green") .title}}{{end}}'

 # format a pull request using multiple tables with headers
 $ gh pr view 3519 --json number,title,body,reviews,assignees --template \
 '{{printf "#%v" .number}} {{.title}}

 {{.body}}

 {{tablerow "ASSIGNEE" "NAME"}}{{range .assignees}}{{tablerow .login .name}}{{end}}{{tablerender}}
 {{tablerow "REVIEWER" "STATE" "COMMENT"}}{{range .reviews}}{{tablerow .author.login .state .body}}{{end}}
 '

Also, you don't need `{{tablerender}}` unless you want to render any text - plain or even another table - after that. The table is automatically rendered after all records are processed if not rendered explicitly first.

How would I add, for example, a comma-separated list of label names into a `tablerow`? I've tried:

`.labels.name`

but get:

template: :1:135: executing "" at : can't evaluate field name in type interface {}

I've also tried various forms of nesting `{{range}}` instead tablerow and using `join` but nothing seems to work. I've got an equivalent jq expression working, but would rather use a go template for better formatting.

Go templates don't expand slices like that. But you can use some scope operator like `range` or a specific operator like `pluck` for a slice of labels like so:

 URL


Question: WebSocket Disconnection Issue After Running for Some Time the Companion Web API Service?

@NoTuxNoBux, the plan proposed here would be to stop releasing the "cloud webapp" container, which would include the Twin API you are referencing. But this companion webapp just wraps the device twin and device methods behind a REST interface (with retries etc.) so apps in the cloud do not need to call any IoT Hub device methods directly. So that you are not seeing the same behavior is surprising and would need to be looked at. You could use the code in the web app in your Azure function, would that work?

You could also use 2.9.14 of the container on MCR, but there would not be any more security fixes published, nor would we run end to end tests with 2.9.15+ of OPC Publisher then.

Anyway, we could slice the webapp to just what you need and make it a sample, if that is ok for you, a list of API you are using would be great.

The core issue is that we just do not have cycles to update the deployment scripts and e2e devops pipelines to comply with security requirements for hosting on Microsoft build system. we are forced to turn off our builds/tests for this component at some point or muster time to stay in compliance.

I want to say this too, so it is not misconstrued: The OPC Publisher is not affected by this at all, we did all the work to be compliant internally already. It can still be controlled via Device Methods from IoT Hub, the API will not change, and no support timelines/conditions are affected, it remains fully integrated there and no changes are planned there. The OPC Publisher module will also retain all its capabilities, nothing will be removed which includes its HTTPS API endpoint if you enable it.

There was very limited feedback here until yesterday, nor did we ever get a support ticket on this component, which indicates that usage is very light. It would be great if we can come up with a plan quickly that does not break anyone.

WebSocket Disconnection Issue After Running for Some Time the Companion Web API Service
**Describe the bug**
We are experiencing an issue where WebSocket connections in our application are getting disconnected after running for some time. The application initially runs without any issues, but after a certain period, WebSockets stop functioning, and we receive the following error in the logs when performing certain API calls:

`
fail: Microsoft.AspNetCore.Server.Kestrel[13]
 Connection id "*****", Request id "*****:00000001": An unhandled exception was thrown by the application.
 {"type":"WebSocketException","title":"HttpRequestException","status":500,"detail":"-2147467259"}
`

After a restart of the service, everything works correctly again.

**To Reproduce**
1. Deploy mcr.microsoft.com/iot/industrial-iot-services-all:2.9.11
2. Perform an API call to get history values: history/v2/read/ /values
3. At the beginning is working properly, after some time it stops working

**Expected behavior**
WebSocket connections should remain stable without unexpected disconnections.

**Actual Behavior**
WebSockets disconnect after a certain period, and the application logs the error mentioned above.

**Version**
The version used is mcr.microsoft.com/iot/industrial-iot-services-all:2.9.11

Do you have more logs to pinpoint the issue? It sounds the entire app service is not working anymore, is that correct? Because you are saying you needed to restart the app service instance/service.

Yes, here I add more logs. The app is still working and replying successfully to other API calls, but not to the history read ones for a given endpoint.

`
fail: Microsoft.AspNetCore.Server.Kestrel[13]
 Connection id "0HNB566KS58R0", Request id "0HNB566KS58R0:00000001": An unhandled exception was thrown by the application.
 {"type":"WebSocketException","title":"HttpRequestException","status":500,"detail":"-2147467259"}
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
 Request finished HTTP/1.1 POST URL - 500 0 - 846.6320ms
info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[105]
 Executed action Azure.IIoT.OpcUa.Publisher.Service.WebApi.Controllers.HistoryController.HistoryReadValuesAsync (Azure.IIoT.OpcUa.Publisher.Service.WebApi) in 884.4223ms
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
 Executed endpoint 'Azure.IIoT.OpcUa.Publisher.Service.WebApi.Controllers.HistoryController.HistoryReadValuesAsync (Azure.IIoT.OpcUa.Publisher.Service.WebApi)'
fail: Microsoft.AspNetCore.Server.Kestrel[13]
 Connection id "0HNB566KS58QV", Request id "0HNB566KS58QV:00000001": An unhandled exception was thrown by the application.
 {"type":"WebSocketException","title":"HttpRequestException","status":500,"detail":"-2147467259"}
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
 Request finished HTTP/1.1 POST URL - 500 0 - 885.1963ms
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
 Request starting HTTP/1.1 GET URL - - -
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
 Request starting HTTP/1.1 GET URL - - -
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
 Executing endpoint 'Health checks'
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
 Executing endpoint 'Health checks'
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
 Executed endpoint 'Health checks'
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
 Executed endpoint 'Health checks'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
 Request finished HTTP/1.1 GET URL - 200 - text/plain 0.4270ms
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
 Request finished HTTP/1.1 GET URL - 200 - text/plain 0.4268ms
`

We are removing the companion app as per #2370 , closing as wont fix.

Notice: Deprecation and removal of cloud REST companion web application is planned for release 2.9.15.
Based on the feedback below that we received since Jan 17th, we will remove the REST companion web app in 2.9.15 release and only ship the OPC Publisher IoT Edge module going forward.

At least for us it is not a concern. Albeit it is always nice to have a sample.

Hi @marcschier Thanks for clarifying, in that case, if I understood correctly, we are indeed not affected by this change and don't mind it going away 🙂.

What we are doing right now is indeed calling HTTPS/REST API methods on the OPC Publisher directly on the IP address of the machine hosting it, using ApiKey authentication. From what I understand now the cloud webapp is a separate application that we thus aren't using (we just deploy the OPC Publisher itself).


Question: Assertion fails if cancellation is requested before connection establish?

All forms of cancellation are now supported, for example, implicit with operator || and && or explicit with the cancel member function.

Assertion fails if cancellation is requested before connection establish
It seems when we send cancellation signal during the connection phase this assertion fails.
i'm using URL
`
: ~/third_party/aedis/include/aedis/detail/connection_ops.hpp:536: void aedis::detail::reader_op >::operator()(Self &, boost::system::error_code, std::size_t) [Conn = aedis::connection<>, Self = boost::asio::detail::composed_op >, boost::asio::detail::composed_work , boost::asio::experimental::detail::parallel_group_op_handler >, boost::asio::detail::composed_work , boost::asio::detail::composed_op >, boost::asio::detail::composed_work , boost::asio::detail::as_tuple_handler >>, void (boost::system::error_code)>, void (boost::system::error_code)>, (lambda at ~/third_party/aedis/include/aedis/detail/connection_ops.hpp:358:13), (lambda at ~/third_party/aedis/include/aedis/detail/connection_ops.hpp:359:13), (lambda at ~/third_party/aedis/include/aedis/detail/connection_ops.hpp:360:13), (lambda at ~/third_party/aedis/include/aedis/detail/connection_ops.hpp:361:13)>, void (boost::system::error_code)>]: Assertion !conn->read_buffer_.empty()' failed.
`

You can easily reproduce it with simple loop like:

C++
for (;;)
{
 auto timer = asio::steady_timer{ executor, std::chrono::milliseconds{ 10 } };
 co_await (connection.async_run(endpoint, asio::use_awaitable) || timer.async_wait(asio::use_awaitable));
}

Thanks for reporting. Aedis was designed to provide built-in timeouts. The call to

co_await (connection.async_run(endpoint, asio::use_awaitable) || timer.async_wait(asio::use_awaitable));

is probably messing around with the internal operations in a way I did not test before. I am considering disabling this type of cancellation unless there is a reason for not doing so. I want to avoid every user implementing timeouts on their side over and over again. For all possible timeouts see URL

Timeout here is just for demonstration, I'm not using timers like that I just cancel the connection for other reasons.
Point is that async_run is not can cancellable properly.

How are you cancelling, with `connection::cancel(operation::run)`? Do you observe the same behaviour if you cancel like this

timer.async_wait(&{conn.cancel(operation::run)})

I will investigate this issue on the weekend.

Haven't test that, but because I am using it in coroutines, it needs to be cancellable because even if I use current coroutines cancellation slot it will get overwritten by `co_await conn.async_run(...)`

I can't reproduce the crash with this

#include
#include

#include
#include
#include
#include

namespace net = boost::asio;
using namespace net::experimental::awaitable_operators;
using aedis::endpoint;
using connection = aedis::connection<>;

net::awaitable reconnect(std::shared_ptr db)
{
 net::steady_timer timer{co_await net::this_coro::executor};
 for (;;) {
 timer.expires_after(std::chrono::milliseconds{10});
 endpoint ep{"127.0.0.1", "6379"};
 co_await (
 db->async_run(ep, net::use_awaitable) ||
 timer.async_wait(net::use_awaitable)
 );
 std::cout (ioc);
 net::co_spawn(ioc, reconnect(db), net::detached);
 ioc.run();
}

The crash on line `: ~/third_party/aedis/include/aedis/detail/connection_ops.hpp:536` is not possible on a normal cancellation. I am afraid this is indicating a buffer overflow somewhere in your program. Can you please enable sanitizers? For example, I get a crash if I use

 co_await (
 db->async_run({"127.0.0.1", "6379"}, net::use_awaitable) ||
 timer.async_wait(net::use_awaitable)
 );

instead of

 endpoint ep{"127.0.0.1", "6379"};
 co_await (
 db->async_run(ep, net::use_awaitable) ||
 timer.async_wait(net::use_awaitable)
 );

I can reproduce the with your code here, it takes a lite time to happen (most of the times under 60 sec). some times it leads to that assertion fail some times dead lock.
Try change the time form 10ms to 6ms it happens faster.
And of course make sure you are in debug build.
Changing timer to something low like 1ms leads to a new assertion fail:
`test: ~/third_party/aedis/include/aedis/detail/net.hpp:101: void aedis::detail::resolve_op , boost::asio::basic_waitable_timer >::operator()(Self &, std::array , boost::system::error_code, boost::asio::ip::tcp::resolver::results_type, boost::system::error_code) [Resolver = boost::asio::ip::basic_resolver , Timer = boost::asio::basic_waitable_timer , Self = boost::asio::detail::composed_op , boost::asio::basic_waitable_timer >, boost::asio::detail::composed_work , boost::asio::detail::composed_op >, boost::asio::detail::composed_work , boost::asio::detail::composed_op >, boost::asio::detail::composed_work , boost::asio::detail::awaitable_handler , void (boost::system::error_code)>, void (boost::system::error_code)>, void (boost::system::error_code, boost::asio::ip::basic_resolver_results )>]: Assertion (!ec2)&&("resolve_op: Incompatible state.")' failed.`

The second assertion you found is something I thought was impossible to happen but does not represent a pre condition for my code to work. I will remove it since it can indeed happen. Thanks again for reporting.

This commit URL fixes the problems I could reproduce so far. There is still a problem though if use very short timeouts like 1ms, in this case it seems that the cancellation gets lost. I will have to investigate more.

Are you using the same commit that I mentioned in first comment?
here is another assertion fail:
`
test: ~/third_party/aedis/include/aedis/resp3/detail/exec.hpp:133: void aedis::resp3::detail::exec_with_timeout_op , boost::asio::basic_waitable_timer , aedis::adapter::detail::general_aggregate >>, boost::asio::dynamic_string_buffer , std::allocator >>::operator()(Self &, std::array , boost::system::error_code, std::size_t, boost::system::error_code) [AsyncStream = boost::asio::basic_stream_socket , Timer = boost::asio::basic_waitable_timer , Adapter = aedis::adapter::detail::general_aggregate >>, DynamicBuffer = boost::asio::dynamic_string_buffer , std::allocator >, Self = boost::asio::detail::composed_op , boost::asio::basic_waitable_timer , aedis::adapter::detail::general_aggregate >>, boost::asio::dynamic_string_buffer , std::allocator >>, boost::asio::detail::composed_work , boost::asio::detail::composed_op >, boost::asio::detail::composed_work , boost::asio::detail::awaitable_handler , void (boost::system::error_code)>, void (boost::system::error_code, unsigned long)>]: Assertion (!ec2)&&("exec_with_timeout_op: Unexpected completion order.")' failed.
`

Maybe using the following code helps to reproduce it faster:

C++

#include
#include

#include
#include
#include
#include

namespace net = boost::asio;
using namespace net::experimental::awaitable_operators;
using aedis::endpoint;
using connection = aedis::connection<>;

net::awaitable
reconnect()
{
 auto executor = co_await net::this_coro::executor;
 auto db1 = connection{ executor };
 auto db2 = connection{ executor };
 auto db3 = connection{ executor };
 auto db4 = connection{ executor };
 auto timer = net::steady_timer{ executor };
 for (int i = 0;; i++)
 {
 timer.expires_after(std::chrono::milliseconds{ 10 });
 endpoint ep{ "127.0.0.1", "6379" };
 co_await (db1.async_run(ep, net::use_awaitable) || db2.async_run(ep, net::use_awaitable) ||
 db3.async_run(ep, net::use_awaitable) || db4.async_run(ep, net::use_awaitable) ||
 timer.async_wait(net::use_awaitable));
 std::cout << i << std::endl;
 }
}

int
main()
{
 net::io_context ioc;
 net::co_spawn(ioc, reconnect(), net::detached);
 ioc.run();
}

I have created a new issue to improve cancellation support, see #32 so I will continue there. AFAICS, this ticket can be closed as the crashes have been fixed.

Improve support for cancellation
At the moment the `connection` classes provide support to cancellation with `cancel` member function. Although not common or even advisable, some users may want to cancel operations with timeouts like this

cpp
co_await (conn.async_run(...) || timer.async_wait(...));

This is working as long as the timeout is not too small e.g. `10ms`. To handle that properly Aedis need to improve cancelation support. Below some parts of a conversation I've had with Richard

cpp
// some code...
asio::post(exec, some_next_step(std::move(my_handler));
// cancellation happens here...
// some_next_step will not know that the handler was cancelled unless it checks.
// ...
// ... some_next_step now executes...

cpp
void some_next_step(auto handler)
{
 auto cs = get_canecllation_state(handler);
 if (cs.cancelled() != asio::cancellation_type::none)
 {
 complete(handler, operation_aborted);
 return;
 }
 // ... not cancelled
}