Course Description
| Course | Code | Semester | T+P (Hour) | Credit | ECTS |
|---|
| DEEP LEARNING for NATURAL LANGUAGE PROCESSING | COED1113992 | Fall Semester | 3+0 | 3 | 8 |
| Prerequisites Courses | |
| Recommended Elective Courses | |
| Language of Course | English |
| Course Level | Third Cycle (Doctorate Degree) |
| Course Type | Elective |
| Course Coordinator | Prof.Dr. Selim AKYOKUŞ |
| Name of Lecturer(s) | Prof.Dr. Selim AKYOKUŞ |
| Assistant(s) | |
| Aim | Natural language processing (NLP) stands out as a pivotal technology in the information age. Its applications influences various aspects of our lives, given that human communication encompasses a wide array of activities: from web searches, advertising, and emails to customer service, language translation, virtual agents, medical reports, and political discourse. Over the past decade, deep learning, employing neural network approaches, has demonstrated remarkable efficacy in numerous NLP tasks. This involves the use of singular end-to-end neural models that eliminate the need for traditional, task-specific feature engineering. This course offers students a comprehensive introduction to the latest advancements in Deep Learning for NLP. Through a combination of lectures, assignments, and a final project, participants will acquire the essential skills to conceptualize, implement, and comprehend different neural network models. |
| Course Content | This course contains; Introduction to NLP and Deep Learning,Foundations of NLP, Machine Learning and Deep Learning,Vector Semantics and Embeddings,Language Models,Neural Networks and Neural Language Models,Recurrent Neural Networks and Language Models,Seq2Seq, Machine Translation, Subword Models,Exam Week,Transformers and Pretrained Language Models ,Transformers,Fine-tuning and Masked Language Models ,Prompting and Instruct Tuning,Question Answering, Chatbots and Dialogue Systems, Automatic Speech Recognition and Text-to-Speech Conversion
,Project presentations. |
| Dersin Öğrenme Kazanımları | Teaching Methods | Assessment Methods |
| Recognizes the natural language processing and adopt advanced natural language processing techniques to solve real-world problems. | 2 | E |
| Analyze state-of-the-art deep learning architectures for NLP. | 16, 2 | D, F |
| Implement the common deep neural network models for NLP. | 12, 14, 21, 6, 9 | A, D, G |
| Evaluate the research literature on the application of deep learning to natural language processing, and prepare a research project with deep learning architectures for natural language processing and summarize its contents through an oral presentation. | 14, 2 | F |
| Teaching Methods: | 12: Problem Solving Method, 14: Self Study Method, 16: Question - Answer Technique, 2: Project Based Learning Model, 21: Simulation Technique, 6: Experiential Learning, 9: Lecture Method |
| Assessment Methods: | A: Traditional Written Exam, D: Oral Exam, E: Homework, F: Project Task, G: Quiz |
Course Outline
| Order | Subjects | Preliminary Work |
|---|
| 1 | Introduction to NLP and Deep Learning | |
| 2 | Foundations of NLP, Machine Learning and Deep Learning | |
| 3 | Vector Semantics and Embeddings | |
| 4 | Language Models | |
| 5 | Neural Networks and Neural Language Models | |
| 6 | Recurrent Neural Networks and Language Models | |
| 7 | Seq2Seq, Machine Translation, Subword Models | |
| 8 | Exam Week | |
| 9 | Transformers and Pretrained Language Models | |
| 10 | Transformers | |
| 11 | Fine-tuning and Masked Language Models | |
| 12 | Prompting and Instruct Tuning | |
| 13 | Question Answering, Chatbots and Dialogue Systems, Automatic Speech Recognition and Text-to-Speech Conversion
| |
| 14 | Project presentations | |
| Resources |
| - Dan Jurafsky and James H. Martin. Speech and Language Processing
- Jacob Eisenstein. Natural Language Processing
- Yoav Goldberg. A Primer on Neural Network Models for Natural Language Processing
- Delip Rao and Brian McMahan. Natural Language Processing with PyTorch
- Lewis Tunstall, Leandro von Werra, and Thomas Wolf. Natural Language Processing with Transformers
|
| Natural Language Processing with Python, Steven Bird, Ewan Klein, and Edward Loper at http://www.nltk.org/book/ |
Course Contribution to Program Qualifications
| Course Contribution to Program Qualifications |
| No | Program Qualification | Contribution Level |
| 1 | 2 | 3 | 4 | 5 |
| 1 | Develop and deepen the current and advanced knowledge in the field with original thought and/or research and come up with innovative definitions based on Master's degree qualifications. | | X | | | |
| 2 | Conceive the interdisciplinary interaction which the field is related with ; come up with original solutions by using knowledge requiring proficiency on analysis, synthesis and assessment of new and complex ideas. | | | X | | |
| 3 | Evaluate and use new information within the field in a systematic approach and gain advanced level skills in the use of research methods in the field. | | | X | | |
| 4 | Develop an innovative knowledge, method, design and/or practice or adapt an already known knowledge, method, design and/or practice to another field. | | | | | |
| 5 | Broaden the borders of the knowledge in the field by producing or interpreting an original work or publishing at least one scientific paper in the field in national and/or international refereed journals. | | | | X | |
| 6 | Contribute to the transition of the community to an information society and its sustainability process by introducing scientific, technological, social or cultural improvements. | | | | | |
| 7 | Independently perceive, design, apply, finalize and conduct a novel research process. | | | | X | |
| 8 | Ability to communicate and discuss orally, in written and visually with peers by using a foreign language at least at a level of European Language Portfolio C1 General Level. | | | | X | |
| 9 | Critical analysis, synthesis and evaluation of new and complex ideas in the field. | | X | | | |
| 10 | Recognizes the scientific, technological, social or cultural improvements of the field and contribute to the solution finding process regarding social, scientific, cultural and ethical problems in the field and support the development of these values. | | | X | | |
Assessment Methods
| Contribution Level | Absolute Evaluation |
| Rate of Midterm Exam to Success | | 50 |
| Rate of Final Exam to Success | | 50 |
| Total | | 100 |
| ECTS / Workload Table |
| Activities | Number of | Duration(Hour) | Total Workload(Hour) |
| Course Hours | 14 | 3 | 42 |
| Guided Problem Solving | 0 | 0 | 0 |
| Resolution of Homework Problems and Submission as a Report | 10 | 2 | 20 |
| Term Project | 0 | 0 | 0 |
| Presentation of Project / Seminar | 8 | 10 | 80 |
| Quiz | 6 | 3 | 18 |
| Midterm Exam | 1 | 30 | 30 |
| General Exam | 1 | 50 | 50 |
| Performance Task, Maintenance Plan | 0 | 0 | 0 |
| Total Workload(Hour) | 240 |
| Dersin AKTS Kredisi = Toplam İş Yükü (Saat)/30*=(240/30) | 8 |
| ECTS of the course: 30 hours of work is counted as 1 ECTS credit. |
Detail Informations of the Course
Course Description
| Course | Code | Semester | T+P (Hour) | Credit | ECTS |
|---|
| DEEP LEARNING for NATURAL LANGUAGE PROCESSING | COED1113992 | Fall Semester | 3+0 | 3 | 8 |
| Prerequisites Courses | |
| Recommended Elective Courses | |
| Language of Course | English |
| Course Level | Third Cycle (Doctorate Degree) |
| Course Type | Elective |
| Course Coordinator | Prof.Dr. Selim AKYOKUŞ |
| Name of Lecturer(s) | Prof.Dr. Selim AKYOKUŞ |
| Assistant(s) | |
| Aim | Natural language processing (NLP) stands out as a pivotal technology in the information age. Its applications influences various aspects of our lives, given that human communication encompasses a wide array of activities: from web searches, advertising, and emails to customer service, language translation, virtual agents, medical reports, and political discourse. Over the past decade, deep learning, employing neural network approaches, has demonstrated remarkable efficacy in numerous NLP tasks. This involves the use of singular end-to-end neural models that eliminate the need for traditional, task-specific feature engineering. This course offers students a comprehensive introduction to the latest advancements in Deep Learning for NLP. Through a combination of lectures, assignments, and a final project, participants will acquire the essential skills to conceptualize, implement, and comprehend different neural network models. |
| Course Content | This course contains; Introduction to NLP and Deep Learning,Foundations of NLP, Machine Learning and Deep Learning,Vector Semantics and Embeddings,Language Models,Neural Networks and Neural Language Models,Recurrent Neural Networks and Language Models,Seq2Seq, Machine Translation, Subword Models,Exam Week,Transformers and Pretrained Language Models ,Transformers,Fine-tuning and Masked Language Models ,Prompting and Instruct Tuning,Question Answering, Chatbots and Dialogue Systems, Automatic Speech Recognition and Text-to-Speech Conversion
,Project presentations. |
| Dersin Öğrenme Kazanımları | Teaching Methods | Assessment Methods |
| Recognizes the natural language processing and adopt advanced natural language processing techniques to solve real-world problems. | 2 | E |
| Analyze state-of-the-art deep learning architectures for NLP. | 16, 2 | D, F |
| Implement the common deep neural network models for NLP. | 12, 14, 21, 6, 9 | A, D, G |
| Evaluate the research literature on the application of deep learning to natural language processing, and prepare a research project with deep learning architectures for natural language processing and summarize its contents through an oral presentation. | 14, 2 | F |
| Teaching Methods: | 12: Problem Solving Method, 14: Self Study Method, 16: Question - Answer Technique, 2: Project Based Learning Model, 21: Simulation Technique, 6: Experiential Learning, 9: Lecture Method |
| Assessment Methods: | A: Traditional Written Exam, D: Oral Exam, E: Homework, F: Project Task, G: Quiz |
Course Outline
| Order | Subjects | Preliminary Work |
|---|
| 1 | Introduction to NLP and Deep Learning | |
| 2 | Foundations of NLP, Machine Learning and Deep Learning | |
| 3 | Vector Semantics and Embeddings | |
| 4 | Language Models | |
| 5 | Neural Networks and Neural Language Models | |
| 6 | Recurrent Neural Networks and Language Models | |
| 7 | Seq2Seq, Machine Translation, Subword Models | |
| 8 | Exam Week | |
| 9 | Transformers and Pretrained Language Models | |
| 10 | Transformers | |
| 11 | Fine-tuning and Masked Language Models | |
| 12 | Prompting and Instruct Tuning | |
| 13 | Question Answering, Chatbots and Dialogue Systems, Automatic Speech Recognition and Text-to-Speech Conversion
| |
| 14 | Project presentations | |
| Resources |
| - Dan Jurafsky and James H. Martin. Speech and Language Processing
- Jacob Eisenstein. Natural Language Processing
- Yoav Goldberg. A Primer on Neural Network Models for Natural Language Processing
- Delip Rao and Brian McMahan. Natural Language Processing with PyTorch
- Lewis Tunstall, Leandro von Werra, and Thomas Wolf. Natural Language Processing with Transformers
|
| Natural Language Processing with Python, Steven Bird, Ewan Klein, and Edward Loper at http://www.nltk.org/book/ |
Course Contribution to Program Qualifications
| Course Contribution to Program Qualifications |
| No | Program Qualification | Contribution Level |
| 1 | 2 | 3 | 4 | 5 |
| 1 | Develop and deepen the current and advanced knowledge in the field with original thought and/or research and come up with innovative definitions based on Master's degree qualifications. | | X | | | |
| 2 | Conceive the interdisciplinary interaction which the field is related with ; come up with original solutions by using knowledge requiring proficiency on analysis, synthesis and assessment of new and complex ideas. | | | X | | |
| 3 | Evaluate and use new information within the field in a systematic approach and gain advanced level skills in the use of research methods in the field. | | | X | | |
| 4 | Develop an innovative knowledge, method, design and/or practice or adapt an already known knowledge, method, design and/or practice to another field. | | | | | |
| 5 | Broaden the borders of the knowledge in the field by producing or interpreting an original work or publishing at least one scientific paper in the field in national and/or international refereed journals. | | | | X | |
| 6 | Contribute to the transition of the community to an information society and its sustainability process by introducing scientific, technological, social or cultural improvements. | | | | | |
| 7 | Independently perceive, design, apply, finalize and conduct a novel research process. | | | | X | |
| 8 | Ability to communicate and discuss orally, in written and visually with peers by using a foreign language at least at a level of European Language Portfolio C1 General Level. | | | | X | |
| 9 | Critical analysis, synthesis and evaluation of new and complex ideas in the field. | | X | | | |
| 10 | Recognizes the scientific, technological, social or cultural improvements of the field and contribute to the solution finding process regarding social, scientific, cultural and ethical problems in the field and support the development of these values. | | | X | | |
Assessment Methods
| Contribution Level | Absolute Evaluation |
| Rate of Midterm Exam to Success | | 50 |
| Rate of Final Exam to Success | | 50 |
| Total | | 100 |
Numerical Data
Ekleme Tarihi: 24/12/2023 - 02:34Son Güncelleme Tarihi: 24/12/2023 - 02:34
×- A-Z Programs
- Undergraduate
- Graduate
- Academic Calendar
- Double Major & Minor Programs
- Erasmus
- Prospective Students
- Registration
- Re-Enrolment
- Fees
- Directorate of Registrar’s Office
- FAQ
- Accommodation
- Scholarships
- Lateral and Vertical Transfer
- Summer School
- Preparation
- Transportation
