eachbatch is not CuArray friendly

[xukai92]

x = cu(rand(2, 10))
x1 = first(eachbatch(x, size=5))
typeof(x1) # => Array{Float32,2}

I would expect to get CuArray{Float32,2} back, or at least a view there. Is there a way to achieve what I want with the existing code base?

[johnnychen94]

taking advantages of MappedArrays, you could do things like:

x_gpu = mappedarray(cu, batchview(x, size=5)); # add ; here to disable display

the conversion from CPU to CUDA is lazily evaluated.

Edit:

Well, looks like batchview(cu(x), size=5) does the same thing here (but not lazy evaluation).

[Evizero]

While the issue has a point, I think its worth pointing out that its more common and (aside from toy examples) more useful to not move all the data to the GPU at once.

Here is an old snippet demonstrating what I mean

for (batch_x_cpu, batch_y) in eachbatch((train_x ,train_y), batchsize)
    batch_x = KnetArray{Float32}(batch_x_cpu)
    g = costgrad(w, batch_x, batch_y)
    Knet.update!(w, g, lr = lr)
end

[xukai92]

Thanks @johnnychen94 - batchview is what I'm looking for. I wonder how should I go through the doc so that I don't miss things like this?

@Evizero Yes I do this too when data is large. But for cases when I prototype my methods on small datasets (MNIST, CIFAR10), I just prefer move them all to the GPU at once to make the loop a bit faster :)

Thanks for both of you again. I'm closing this issue now.