Remaining problems with breaking changes WGPU 24 -> WGPU 30.

[John-Nagle]

At start, I had 150 compile errors from breaking changes in WGPU. I've fixed most of them, and now I'm down to 5. Some of these are a bit tricky. A bit of help would be appreciated.

This is all related to updating my fork of Rend3, which I have to keep alive for my main projects. This is on branch "wgpu30test", and you can build this from Github if you like.

error[E0614]: type `QueueWriteBufferView` cannot be dereferenced
  --> rend3/src/util/buffer.rs:66:41
   |
66 |         encase::StorageBuffer::new(&mut *mapped).write(data).unwrap();
   |                                         ^^^^^^^ can't be dereferenced

This is related to the new guards added to buffer access, documented in the WGPU change log. I'm not quite sure how to work around this change.

error[E0599]: no method named `pop_error_scope` found for reference `&'a wgpu::Device` in the current scope
   --> rend3/src/util/error_scope.rs:37:38
    |
 37 |         let mut future = self.device.pop_error_scope();
    |                                      ^^^^^^^^^^^^^^^
    |
help: there is a method `push_error_scope` with a similar name, but with different arguments
   --> /home/john/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wgpu-30.0.0/src/api/device.rs:542:5
    |
542 |     pub fn push_error_scope(&self, filter: ErrorFilter) -> ErrorScopeGuard {
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This is just a consequence of error scope being made thread-local. I'll work that out.

error[E0599]: no method named `present` found for struct `wgpu::SurfaceTexture` in the current scope
  --> rend3/src/util/output.rs:27:21
   |
27 |             surface.present();
   |                     ^^^^^^^ method not found in `wgpu::SurfaceTexture`

Per the changelog, "present()" was made a function of Queue rather than Surface. The problem is that the caller doesn't have access to Queue at that point, and that function is exported beyond Rend3 into applications. So this breaking change can cascade through levels. Is there some easy way to get Queue from the available info?

error[E0308]: mismatched types
   --> rend3/src/util/scatter_copy.rs:103:65
    |
103 |         let mapped_slice: &mut [u32] = bytemuck::cast_slice_mut(&mut mapped_range);
    |                                        ------------------------ ^^^^^^^^^^^^^^^^^ expected `&mut [_]`, found `&mut Result<BufferViewMut, ...>`
    |                                        |
    |                                        arguments to this function are incorrect
    |
    = note: expected mutable reference `&mut [_]`
               found mutable reference `&mut std::result::Result<BufferViewMut, MapRangeError>`
note: function defined here
   --> /home/john/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/bytemuck-1.25.0/src/lib.rs:419:8
    |
419 | pub fn cast_slice_mut<
    |        ^^^^^^^^^^^^^^

error[E0608]: cannot index into a value of type `std::result::Result<BufferViewMut, MapRangeError>`
  --> rend3/src/util/upload.rs:59:20
   |
59 |             mapping[upload.staging_offset as usize..][..upload.data.len()].copy_from_slice(upload.data);
   |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Those last two are related to the new buffer access guards, as mentioned above.

I'm trying to do this with minimal changes, cautiously. So I'm asking for advice. Especially around the buffer access area, which involves such touchy things as write-only slices. I don't want to break things that are close to raw memory access.

[kpreid]

error[E0614]: type `QueueWriteBufferView` cannot be dereferenced
  --> rend3/src/util/buffer.rs:66:41
   |
66 |         encase::StorageBuffer::new(&mut *mapped).write(data).unwrap();
   |                                         ^^^^^^^ can't be dereferenced

This is related to the new guards added to buffer access, documented in the WGPU change log. I'm not quite sure how to work around this change.

You must use the methods of wgpu::WriteOnly to write data. It is necessarily impossible to obtain an &mut [u8] here.

For encase, it looks like you may be able to write a wrapper type around wgpu::WriteOnly which implements encase::internal::BufferMut, and that will enable you to continue using encase::StorageBuffer.

error[E0599]: no method named `present` found for struct `wgpu::SurfaceTexture` in the current scope
  --> rend3/src/util/output.rs:27:21
   |
27 |             surface.present();
   |                     ^^^^^^^ method not found in `wgpu::SurfaceTexture`

Per the changelog, "present()" was made a function of Queue rather than Surface. The problem is that the caller doesn't have access to Queue at that point, and that function is exported beyond Rend3 into applications. So this breaking change can cascade through levels. Is there some easy way to get Queue from the available info?

All wgpu objects are now clonable, so you can clone the Queue at some earlier point and store it for use at this point.

[John-Nagle]

Thanks. Now I'm down to two compile errors, both revolving around "encase" vs. write only buffers.

error[E0308]: mismatched types
   --> rend3/src/util/scatter_copy.rs:103:65
    |
103 |         let mapped_slice: &mut [u32] = bytemuck::cast_slice_mut(&mut mapped_range);
    |                                        ------------------------ ^^^^^^^^^^^^^^^^^ expected `&mut [_]`, found `&mut Result<BufferViewMut, ...>`
    |                                        |
    |                                        arguments to this function are incorrect
    |
    = note: expected mutable reference `&mut [_]`
               found mutable reference `&mut std::result::Result<BufferViewMut, MapRangeError>`

That's trying to get a raw slice of u32 into which it can write, with that slice in a write-only buffer. As you pointed out, that's now a prohibited operation. It's also playing games with accessing a buffer as u32, and I'm not sure how that plays with the new interface.

https://github.com/John-Nagle/rend3-hp/blob/5480169a3dad0092b8b8da18429358177f3a0470/rend3/src/util/scatter_copy.rs#L73

Similarly,

error[E0614]: type `QueueWriteBufferView` cannot be dereferenced
  --> rend3/src/util/buffer.rs:66:41
   |
66 |         encase::StorageBuffer::new(&mut *mapped).write(data).unwrap();
   |                                         ^^^^^^^ can't be dereferenced

https://github.com/John-Nagle/rend3-hp/blob/5480169a3dad0092b8b8da18429358177f3a0470/rend3/src/util/buffer.rs#L61

The previous message:

For encase, it looks like you may be able to write a wrapper type around wgpu::WriteOnly which implements encase::internal::BufferMut, and that will enable you to continue using encase::StorageBuffer.

covers the general idea, but I'd like to get this right the first time, because it's so low-level that it will be a huge pain to debug if there's anything wrong. Detailed advice appreciated. Thanks.

Should there be more WGPU API support for this kind of data moving? Everybody needs to put data in buffers.

[kpreid]

That's trying to get a raw slice of u32 into which it can write, with that slice in a write-only buffer. As you pointed out, that's now a prohibited operation. It's also playing games with accessing a buffer as u32, and I'm not sure how that plays with the new interface.

Use WriteOnly::into_chunks() to get WriteOnly<[[u8; 4]]>, and u32::to_ne_bytes() to turn the u32s into [u8; 4] to write.

but I'd like to get this right the first time, because it's so low-level that it will be a huge pain to debug if there's anything wrong. Detailed advice appreciated.

It does not have to be a pain to debug. You can create a WriteOnly<[u8]> from a &mut [u8] using WriteOnly::from_mut(), and so you can write plain Rust unit tests for the wrapper that show that it produces the right byte sequence.

Should there be more WGPU API support for this kind of data moving?

It would be nice if all this was easier, certainly. The most straightforward way to do it would be for wgpu to implement encase’s traits, and bytemuck’s, and so on, but that would only work for current versions of those libraries, and generally be additional dependency management hassle. Anything in that direction should be written up as an issue to hash out the costs, benefits, and design considerations.

[John-Nagle]

Still plugging away on this. There's wgpu template and trait cleverness, encase template and trait cleverness, bytemuck template and trait cleverness, and rend3 template and trait cleverness. They don't play well together. Wgpu and encase don't know about each other. Wgpu uses bytemuck for its conversions.

WriteOnly probably should have been a lower level concept in some standard library, used by all relevant crates. But that's a messy retrofit.

I need more sleep before attacking this again.

[John-Nagle]

Everything compiles, now working through validation errors of the Rend3 unit tests. Many tests fail with:

thread 'egui::test' (1129698) panicked at /home/john/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wgpu-30.0.1/src/backend/wgpu_core.rs:1148:26:
wgpu error: Validation Error

Caused by:
  In Device::create_bind_group_layout, label = 'D2 texture bgl'
    Too many bindings of type BindingArrayElements in Stage ShaderStages(FRAGMENT), limit is 0, count was 131072. Check the limit `max_binding_array_elements_per_shader_stage` passed to `Adapter::request_device`

I've tried setting, in Limits,

max_binding_array_elements_per_shader_stage: 500000, /* new in WGPU 30*/

The device allocates and opens, but fails at bind group creation. In the error message, that limit is shown as 0.
So I probably have to enable some feature. But which one?

What seems to be going on is that, at WGPU 30, you only get the features you explicitly ask for when opening the adapter. Previously, you could ask for no features, and the adapter gave you everything it offered, which you could then check. There may be some additional feature I need to turn on for this to work. I tried requesting some more features:

        //  New for WGPU 30
        | Features::BUFFER_BINDING_ARRAY.bits()
        | Features::TEXTURE_BINDING_ARRAY.bits()
        | Features::STORAGE_RESOURCE_BINDING_ARRAY.bits()

but it didn't help.

What am I missing here?