Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Good catch - this is actually only for functions defined inside interactive Notebook or iPython environments, and we do have an option to bypass it (which serializes the function and state), but you probably don't want it by default. Any function defined inside normal python (even if you're using it within a notebook) doesn't doesn't need imports or variables in scope, you can ship it as is to remote compute. But notebooks are gnarly things, where you might have defined a variable and changed it many times, and finally used it inside a function which you want to run remotely as a service. You probably don't want that variable's state to be shipped over, and PyTorch doesn't do this either. This is why so much of stateful PyTorch is meant to be defined inside new nn.Module classes, to neatly package the state to send over to GPU memory. We offer more flexibility than that, but the more state you ship over, the more likely you are to hit version conflicts between the local and remote environments, which can be really annoying. We practically never run into those issues at Runhouse nowadays because we think we've found a sweet spot.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: