{{actor}} deleted this content .
-
You could copy the data to a new Tensor instead. You could also transfer the ownership of the Pointer to the new Tensor by providing a deallocator to from_blob().
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
8 replies
-
Thanks! Your new commit worked with Loader.addressof("free"). I can now call retainReference on both the tensor and the pointer, then just close the tensor to close the data as well. Out of curiosity, how would we pass directly values like Pointer.NativeDeallocator.deallocatorAddress since it is a private field? Would we have to use reflection to pull the field out, or is there a better way?
Finally, I attached how I am using your commit below, let me know if I missed anything.
Thanks again!
public static void main(String[] args) {
Tensor tensor1, tensor2;
FloatPointer hidden1, hidden2;
float[] buf = new float[3];
float[] data = new float[]{3,2,1};
try (PointerScope scope = new PointerScope()) {
hidden1 = new FloatPointer(data);
hidden2 = new FloatPointer(data);
tensor1 = torch.for_blob(hidden1).deleter(Loader.addressof("free"))
.make_tensor();
tensor2 = torch.for_blob(hidden2).make_tensor();
tensor1.retainReference();
tensor2.retainReference();
hidden1.retainReference();
hidden2.retainReference();
}
hidden1.get(buf);
System.out.println("tensor1's data before cleanup:" + Arrays.toString(buf));
hidden2.get(buf);
System.out.println("tensor2's data before cleanup:" + Arrays.toString(buf));
tensor1.close();
tensor2.close();
hidden1.get(buf);
System.out.println("tensor1's data after cleanup:" + Arrays.toString(buf));
hidden2.get(buf);
System.out.println("tensor2's data after cleanup:" + Arrays.toString(buf));
//tensor1's data before cleanup:[3.0, 2.0, 1.0]
//tensor2's data before cleanup:[3.0, 2.0, 1.0]
//tensor1's data after cleanup:[0.0, 0.0, -1.0844938E-19]
//tensor2's data after cleanup:[3.0, 2.0, 1.0]
}
Beta Was this translation helpful? Give feedback.
-
Right, it wasn't intended to be used that way, but I don't see how adding getters could harm anything, so I've done that in commit bytedeco/javacpp@730ae11. With that, it can be accessed this way now:
class MyFloatPointer extends FloatPointer {
long deallocatorAddress() {
return ((NativeDeallocator)deallocator()).deallocatorAddress();
}
}
If you want to keep using free() though, that's only guaranteed to work with memory allocated by malloc() and friends, so in that case you should use Pointer.malloc() to allocate memory, as done in the samples for Triton Inference Server: https://github.com/bytedeco/javacpp-presets/tree/master/tritonserver/samples
BTW, call deallocate(false) instead of retainReference(). With the latter, the deallocator may still get called by the GC.
Beta Was this translation helpful? Give feedback.
-
You are right, I noticed my Tensor objects were getting garbage collected when I expose them outside Java (i.e. to Python for bytedeco/javacpp#530), but they worked okay in Java. I ended up setting org.bytedeco.javacpp.nopointergc to true and that solved it.
I took a look at deallocate(false) and logically that makes sense what it would do, but in practice, this didn't quite work for torch's tensor object, as this makes the data_ptr object inaccessible (NullPointerException: This pointer address is NULL). The Tensor is still fine if I call deallocate(false) on hidden and retainReference() on tensor and I can operate with it. I can even call item_float() to look at the first element which looks okay, but I cannot get the result out to Java by accessing its data_ptr_float(). Only way I found to get the data out is if I call torch.clone(tensor2).data_ptr_float(), which shows me that it is working as intended (data is okay outside scope) but not really suitable for my usecase.
Let me know if I have missed something, but if not, it is still not a problem for me because using nopointergc works well for me.
Beta Was this translation helpful? Give feedback.
-
You need to call deallocate(false) on the original FloatPointer object that has a deallocator. It's not going to work if you call it on the Tensor or on Tensor.data_ptr().
Beta Was this translation helpful? Give feedback.
-
It's actually also necessary to call retainReference() since it doesn't remove the deallocator, but only disables it for the GC, so close() can still call it.
Beta Was this translation helpful? Give feedback.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4