For potential applications, please introduce yourselves at The Introductions Thread! and not on this thread. Let’s keep this focused on GSoC ideas and discussions
Google Summer of Code 2024 Ideas
Hello, my name is Amit Subhash Chejara and I am learning machine learning and PyTorch. I have completed my BSc last year. I am interested in two ideas, protein language modeling and torch compile and PyTorch. Since I am currently learning pytorch and have some experience in linear regression models and classification models in pytorch, I want to ask weather I can contribute to this project or I need a more deeper knowledge of pytorch?
Please Guide me!
Hello DeepChem Team,
I am Suraj Mahapatra, a third-year B.Tech student at SRM Institute of Science and Technology. Currently engaged in a research and development program focusing on Large Language Models (LLMs) at NIT Rourkela, I am eager to leverage my skills in the field of Deep Learning and develop successful models out of it. I am writing to express my strong interest in contributing to the development of the Protein Language Model and Improving Equivariance Support.
Previously I have been working on the Hugging Face Model, Vision Transformers, Tensor-flow, PyTorch and I am excited to share my expertise by contributing in your project. Before taking the plunge, I would value your suggestion and mentoring in the project.
Hello! I am Divyanshu Rana, A 1st year student at Graphic era University, where I am pursuing a degree in Master of Computer Applications (MCA). I am really excited about Improving Antibody Support and Layer Tutorials. I have familiarity with deep learning frameworks such as PyTorch that would be essential for implementing antibody-specific models.
Thank you for your attention, consideration, and ongoing support. Together, let us continue to strive for excellence and make a positive difference in the lives of individuals
With warm regards,
Divyanshu Rana
Hello Bharath,
I am Sasidharan, a final-year B.E. student studying Computer Science. I am intrigued by Layers Tutorials, Improving Antibody Support, and Improving New Drug Modality Support, and I’m enthusiastic about contributing my expertise to @DeepChem. Before delving deeper, I would greatly appreciate your guidance on how to get started.
Thank you!
To repeat folks, please don’t introduce yourselves on this channel. It’s not the right place. Please use the introductions channel: The Introductions Thread!
Hello @bharath,
I have been studying the deepchem codebase recently. The community have done quiet an advancement in software aspect in this domain. But I think there is a vacancy in polymers domain. I could not find much of code or contributions on this field. Even in molecule net I could not find much regarding polymers. If there are codes for studying monomers or polymer behaviours or any dataset please help me find it else if it’s suitable for you we can come up with a proposal for a project for GSoC’24. I have few ideas we could discuss.
Hey there! 
I’m Aparna, a third year student at IIT BHU and I’m super excited about the DeepChem Layer Tutorials Enhancement project! I’ve been working on Jupyter Notebook and Colab projects, diving into different machine learning concepts. The project “Layer Tutorials” aligns perfectly with my interests, as I’m eager to improve technical communication skills and contribute to the community. Could you guide me on the best way to get started? I’m ready to dive in and make some meaningful contributions! 
A proposal about polymers could be very welcome. You should try to center it around applications to drug discovery. Come by office hours to discuss with us!
Come by office hours and we would be glad to give guidance!
I am thrilled to express my interest in the project “Protein Language Modeling” for GSOC 2024. My name is Awnish Singh, a fourth-year undergraduate student at BITS Pilani, where I have been deeply involved in research projects under the guidance of Dr. S. Murugesan, focusing on target drug prediction. My experience spans various domains, from computer vision to software development, and I have actively contributed to GSOC in the past.
I am particularly excited about the opportunity to extend DeepChem’s support for using language models with chemistry applications to include protein language modeling. Given the growing importance of protein machine learning in both academia and startups, I believe this project offers a unique opportunity to make a meaningful contribution to the field.
Currently, I am gaining valuable experience through an internship where I am working on power automation with MS Azure and the EasyOCR library for text extraction. My previous involvement in a Genetic Algorithm-based project focused on implementing deep learning models for identifying features of protein coding genes has provided me with a strong foundation in this area.
I am eager to collaborate with the DeepChem community, contribute to the development of a protein language model. I am committed to attending office time meetings and engaging in discussions to ensure the success of this project.
@awnish10-scs Please post in the introductions thread The Introductions Thread!. This channel is only for general GSoC questions about topics
Protein Language Models
- hugging face models
Regarding some application of generating proteins, the training of these models is an expensive task. Usually, transformer type architectures are used for big datasets around 150 million or more sequences. Deepchem offers a huggingface wrapper that can be used to load pre-trained models. Now it is possible to use mask language modeling to fill certain sequences. However, fine tunning needs a considerable amount of RAM: An example in https://github.com/deepchem/deepchem/issues/3838#issuecomment-1956970012
Other nice use of LLM is to produce conditional generation https://www.nature.com/articles/s41587-023-02115-w and would be a nice application. Models like protGTP2 in hugging face offers this capability https://huggingface.co/nferruz/ProtGPT2
Finally, other application of this LLM can be the extraction of embedding that will be used in ML models. One example is (# Low-N protein engineering with data-efficient deep learning) https://www.nature.com/articles/s41592-021-01100-y which uses UniRep, a big protein LLM, to generate features for a supervised learning task. A hugging face model with that option is protein BERT. https://huggingface.co/GrimSqueaker/proteinBERT https://github.com/nadavbra/protein_bert/tree/master
Other models that are not present in hugging face but can be of interest. https://www.nature.com/articles/s41587-022-01618-2#code-availability
Other examples for Protein structure generation. https://huggingface.co/spaces/simonduerr/ProteinMPNN
Antibody support.
A couple of definitions to start the discussion are :
- an antibody? - an immunoprotein responsible for specifically recognizing and binding to potentially pathogenic molecules.
- an antigen? - the molecule that the antibody targets.
Some problems that can be studied in antibody design are structural: For example, the accurate modeling of Antibody-Antigen pairs. Specially in the interaction spots. https://www.sciencedirect.com/science/article/pii/S0959440X22000586. For this kind of task is important to have structural databases such as:
-
https://opig.stats.ox.ac.uk/webapps/sabdab-sabpred/sabdab/about
Once the dataset is selected, the models used for this should be sequence-to-structure or structure-to-structure. The featurization can be processing of sequences or coordinates.
The other issue that can be studied is the binding affinity between the antigen and antibody. This supervised problem needs that affinity value for a pair antibody-antigen.
One of the biggest databases is https://life.bsc.es/pid/skempi2/database/summary. However, other articles used processed versions. For example, https://biosig.lab.uq.edu.au/csm_ab/datasets (CSM-AB: graph-based antibody-antigen binding affinity prediction and docking scoring function) which uses a graph signature as feature. Other examples are https://www.nature.com/articles/s42004-023-01037-7#Sec11
Finally, Other models can use only sequences and are made for optimization of other properties such as aggregation or pharmacokinetics. A full set of models-databases and future perspectives can be found at:
https://www.cell.com/trends/pharmacological-sciences/fulltext/S0165-6147(22)00279-6
New Emerging Drug Modalities.
For this type of functionalities, datasets are crucial. For PROTACS and macrocycle, some featurizer already work. Therefore, some databases of interest can be:
PROTACT-DB
http://cadd.zju.edu.cn/protacdb/help
Macrocycles.
Found this article with an analysis of existing literature.
https://pubs.acs.org/doi/epdf/10.1021/acs.jmedchem.3c00134
2 Likes
Hi, I’m Pranjal Verma, a 3rd year student from IITDelhi. I have gone through the description of Protein Language Modelling and found it really fascinating. I am have done projects in LLMs and hugging face. I would be very excited in contributing to this project. I would highly appreciate your guidance. I am ready to get started. Please let me know if I can communicate with you all regarding this project.
A quick reminder this thread is only for discussions about project ideas and not introductions. Please introduce yourselves on The Introductions Thread!
Hello,
My name is Sherry, and I’m currently pursuing my Ph.D. at the School of Computer Science at Zhejiang University, where I’m in my second year. My research is focused on AI4Science, a fascinating field that blends artificial intelligence with scientific discovery.
I have hands-on experience in protein design, which is reflected in my ongoing research and a manuscript I’m preparing for submission. My previous work was grounded in the SE3Diffusion framework, a model I’m intimately familiar with, both in terms of its codebase and underlying principles.
I wholeheartedly agree with the significance of equivalence in biomolecular representation, a concept that resonates with my research interests. I’m particularly excited about the project aimed at improving equivalence support, and I’m eager to contribute to the DeepChem community.
I’m looking forward to exploring how my expertise can align with the goals of this project and to potentially collaborate with like-minded individuals who share a passion for pushing the boundaries of scientific research through AI.
Best regards,
Sherry
Hi, I am a PhD student interested in Protein Language Modeling and Improving New Drug Modality Support, 2 projects. Let me know what should I need to do.
Thanks.
Regards,
Keya
Hi, I am a third-year undergraduate student from IIT Kharagpur interested in Protein Language Modeling and Improving New Drug Modality Support, 2 projects. How am I supposed to go about contributing.
Thanks.
We recommend joining the discord (https://discord.gg/RYTrUY8Ssn) and coming by the office hours Announcing the DeepChem Office Hours. We can answer general questions on how to get started there
Hello, I am Marija. I am a Machine Learning Researcher, working full time, who would like to join an open source initiative. I have been familiar with GSOC since my time as a PhD student, but I never participated in it. I looked there now to find a suitable project to begin my journey in open source. I would like to implement a wishlist model for your library. Based on what I’ve read in your GitHub issues and documentation, you are looking to implement several algorithms which I am also interested in: 1) E3, 2) DeepONet, 3) Neural ODEs, 4) GenAI for Proteins and Sequences (https://www.nature.com/articles/s41587-023-02115-w), and 5) LLM for proteins (https://www.nature.com/articles/s41587-022-01618-2). Could you let me know which of these projects are still available? I am most interested in the last two projects. Additionally, I would like to know if I am eligible to participate through GSOC. If not, is it possible to get the support of a mentor and to create a timeline for this project? I’ll come to the office hours tomorrow to discuss more details.