[Reaching out to mentors] Interest in Project 5 "OpenVINO AI PC Model Training Kit"

[jellllly420]

Hello Shivam and Aishwarye,

I hope this message finds you well. My name is Zejun Zhao, a first-year M.Phil student in Computer Science at Peking University. I'm very excited about the "OpenVINO AI PC Model Training Kit" project, as it presents a unique opportunity to learn about AI infrastructures from industry experts like you and contribute to the open-source AI community at the same time.

Recently, I've been experimenting with OpenVINO and submitted a prerequisite PR (#29291) regarding PyTorch OP support, which I believe prepares me well for contributing to the proposal. To ensure we're aligned and can collaborate effectively on a strong proposal and project, I'd like to discuss your vision and ideas about this project. Here are some questions on which I'd like to hear about what you think:

• What is the expected end-user experience for our training kit?

  This is my primary concern, as it directly impacts usability and development effort. Should it integrate seamlessly as a PyTorch backend (like cpu and cuda), or function as an external dependency for model transformation?

  The "backend" approach offers better integration with the PyTorch ecosystem and long-term maintainability through upstream merging. However, it poses significant development challenges, particularly in coordinating between communities. As a reference, PyTorch's initial xpu support ([RFC] Intel GPU Runtime Upstreaming pytorch/pytorch#114842) has taken over a year.

  Alternatively, an "external dependency" approach, involving an out-of-tree PyTorch wrapper library for graph modification and model recompilation, may make the proposal more feasible for a GSoC project. This would require even less user code modification but come at the cost of potentially limiting OpenVINO's optimization capabilities. From a user perspective, it would resemble:

  import torch
  import ov_train_kit as ovtk
  
  model = MyModel() # User-defined model with pure PyTorch, free of additional modification.
  transformed_model = ovtk.transform_model(model) # Here we do the graph modification and model recompilation
  
  # Regular training loop using `transformed_model`

  A small user survey among my DL-focused peers indicates that both approaches are desirable, with minimal code modification being the key factor. I'd appreciate your perspective on this decision, or any alternative proposals you might have, to ensure we're aligned moving forward.

• What is the envisioned workflow for integrating model training with OpenVINO inferencing using our training kit?

  As I understand it, regardless of the training kit's design, we'll still leverage PyTorch's training facilities and train a PyTorch model. While AI PC hardware and OpenVINO optimizations, with training-time NNCF quantization, can enhance performance, deploying the trained model for OpenVINO inferencing still requires conversion to OpenVINO IR files. Is this the intended workflow?

• What is the intended relationship between our training kit and existing XPU backend for the DL framework, e.g., PyTorch's xpu backend?

  I'm curious about the positioning of our project within the broader AI PC ecosystem. Ideally, it would synergize with existing XPU runtime support to maximize AI PC potential and leverage PyTorch's capabilities. What are your thoughts on this integration, or do you have any existing plans to share?

• Which framework(s) – PyTorch, TensorFlow, or scikit-learn – should we prioritize, or are we targeting all three?

  The idea page mentions all three frameworks, so I'd like clarification on our focus. I'm personally most familiar with PyTorch and scikit-learn but willing to learn about TensorFlow too. What are your thoughts on this prioritization?

I'd appreciate your insights on these questions, and any additional guidance you may have, to better prepare me for this project. Any relevant materials would also be helpful. If possible, I'd be grateful for the opportunity to develop my proposal in collaboration with your feedback.

Thank you for your time and guidance. I look forward to hearing from you soon and discussing how we can move forward with this project.

Regards,
Zejun Zhao

[jellllly420]

@adrianboguszewski Could you please help me connect with the mentors? Thank you!

[adrianboguszewski]

@sbasia @omeraish

[sbasia]

Hi, @jellllly420, Thanks for your detailed questions.
Please find my response below:
(i)What is the expected end-user experience for our training kit?
->Ideally, we wanted the first approach, and as I explained in the reply( #29389 (comment)), using some of the already enabled support can help in this. However, if this is still not possible in the GSoC timeline, we can definitely go for the 2nd approach.
(ii)What is the envisioned workflow for integrating model training with OpenVINO inferencing using our training kit?
-> The idea is to have the IR files generated during the training phase itself.
(iii)What is the intended relationship between our training kit and the existing XPU backend for the DL framework, e.g., PyTorch's xpu backend?
->Currently, the existing XPU backend support is only for inferencing an already pre-trained model. We want that support to happen during the training phase only.
(iv) Which framework(s) – PyTorch, TensorFlow, or scikit-learn – should we prioritize, or are we targeting all three?
->Let's start with PyTorch as it already has a lot of support enabled which will be easiest to integrate. If time permits, we can integrate other frameworks as well.