Introducing self-organizing neuro-fuzzy Q-networks

Trained with unsupervised learning and finished with an offline model-free fuzzy reinforcement learning approach called fuzzy conservative Q-learning (FCQL), this approach offers more effective and interpretable policies than deep neural networks, while facilitating human-in-the-loop design and explainability.

Code demonstrating our proposed method is publicly available.

Read Now Ask a question

Star FCQL

☕️

John Wesley Hostetter

Computer Science Ph.D. Student

North Carolina State University

John Wesley Hostetter is the owner and developer of PySoft - a flexible Python Soft Computing library, as well as a Ph.D. student of Computer Science at North Carolina State University since the Fall of 2019. He received his Bachelor’s and Master’s degrees in Computer Science at Penn State and NC State University, respectively.

His research interests are concerned with how a mathematical field studying impreciseness called fuzzy logic may aid in re-imagining function approximation in artificial intelligence. More specifically, he develops alternatives to artificial neural networks that are more interpretable called self-organizing neuro-fuzzy networks. These self-organizing neuro-fuzzy networks have the same power as artificial neural networks, but can explain their decision-making. With his adviser, Dr. Min Chi, they leverage this novel and interpretable function approximation technique to improve undergraduate STEM education.

Mr. Hostetter has worked as a graduate teaching assistant in courses such as: Java Programming Concepts; Automata, Grammars & Computability; Automated Learning & Data Analysis. He now works as a teaching assistant for two courses at NC State’s Artificial Intelligence Academy: Introduction to Artificial Intelligence & Machine Learning, in addition to his graduate research assistant role.

Skills

Soft Computing

Fuzzy sets, fuzzy logic, rough sets, approximate reasoning, impreciseness, vagueness, uncertainty

Machine Learning

Supervised/unsupervised/reinforcement learning

Python

Experience with PyTorch, Optuna, iGraph, etc.

Experience

Graduate Research Assistant

North Carolina State University

January 2021 – Present Raleigh, NC

Create novel systematic design processes for self-organizing offline fuzzy reinforcement learning.

Teaching Assistant

Artificial Intelligence Academy

July 2022 – Present Raleigh, NC

Assist in teaching workshops about introductory topics within artificial intelligence and machine learning.

Graduate Teaching Assistant

North Carolina State University

August 2021 – December 2021 Raleigh, NC

CSC 422/522 Automated Learning and Data Analysis (68 students enrolled in CSC 422; 150 students enrolled in CSC 522), an introduction to concepts and methods for extracting knowledge or other useful forms of information from data. Proctored exams, designed homework, prepared solutions, created rubrics, graded and held office hours.

Graduate Research Assistant

North Carolina State University

January 2021 – August 2021 Raleigh, NC

Knowledge distillation of trained artificial neural networks and the development of temporal granule pattern mining.

Graduate Teaching Assistant

North Carolina State University

August 2020 – December 2020 Raleigh, NC

CSC 422/522 Automated Learning and Data Analysis (45 students enrolled in CSC 422; 107 students enrolled in CSC 522), an introduction to concepts and methods for extracting knowledge or other useful forms of information from data. Proctored exams, designed homework, prepared solutions, created rubrics, graded and held office hours.

Graduate Service Assistant

North Carolina State University

May 2020 – August 2020 Raleigh, NC

Developed workshops for the Artificial Intelligence Academy’s Data Mining course offered by NC State.

Graduate Teaching Assistant

North Carolina State University

January 2020 – May 2020 Raleigh, NC

CSC 333 Automata, Grammars, and Computability (69 students enrolled), an introductory course for the theory of computation. Reviewed homeworks/exams for correctness, created grading rubrics and teacher solutions, graded student submissions, and proctored exams. Held office hours weekly to assist students’ learning, and answered student questions on Piazza as well.

Graduate Teaching Assistant

North Carolina State University

August 2019 – December 2019 Raleigh, NC

Supervised and taught two programming lab sessions (23 students in each) of CSC 216 Programming Concepts - Java. Held office hours to help answer students’ questions, graded coursework/projects, and proctored exams.

Application Developer Intern

Global Data Consultants IT Solutions

June 2018 – August 2018 Mechanicsburg, PA

Collaborated in the development of an application for both iOS and Android platforms using Xamarin Forms to assist the company’s clients in approving/rejecting timesheets of contractors.

Scholarships, Fellowships, Grants & Awards

Academic Enhancement Grant

NC State University Aug 2023 – Jan 2024

NCSU Tuition/Fees Grant

NC State University Aug 2023 – Jan 2024

Student Travel Grant

NC State University Computer Science Department Mar 2023

AAMAS Student Scholarship

Autonomous Agents and Multiagent Systems (AAMAS) Conference Mar 2023

Summer Graduate Fellowship

NC State University College of Engineering Jun 2022

A prestigious award only offered to a “small number” of graduate students in recognition of their accomplishments.

Summer Graduate Merit Award

NC State University College of Engineering Jun 2021

A prestigious award only offered to a “small number” of graduate students in recognition of their accomplishments.

Graduate Student Support Plan

NC State University College of Engineering Aug 2019 – May 2020

Financial support for Ph.D. study at NCSU in the form of a stipend and tuition waiver, contingent on satisfactory academic progress

Fellowship Graduate Merit Award

NC State University College of Engineering Jun 2019

A prestigious award only offered to a “small number” of graduate students in recognition of their accomplishments.

Dean’s Scholarship

Embry-Riddle Aeronautical University Scholarship Committee Aug 2015

Selected from a highly qualified applicant group and awarded as a tribute to my academic preparation and performance.

Gallery

Featured Publications

John Wesley Hostetter, Cristina Conati, Xi Yang, Mark Abdelshiheed, Tiffany Barnes, Min Chi

September, 2023 In IVA

XAI to Increase the Effectiveness of an Intelligent Pedagogical Agent

Personalized explanations can significantly improve student learning.

John Wesley Hostetter, Min Chi

August, 2023 In IEEE

Latent Space Encoding for Interpretable Fuzzy Logic Rules in Continuous and Noisy High-Dimensional Spaces

A simple and quick alternative to the famous Wang-Mendel Method for fuzzy logic rule generation is proposed and investigated; it is called the Latent Lockstep Method.

John Wesley Hostetter, Mark Abdelshiheed, Tiffany Barnes, Min Chi

August, 2023 In IEEE

Leveraging Fuzzy Logic Towards More Explainable Reinforcement Learning-Induced Pedagogical Policies on Intelligent Tutoring Systems

We successfully distilled knowledge from a pedagogical policy, represented with a deep neural network, to a shallow neural network, and then mathematically translated this model to a fuzzy system. The fuzzy system was then interpreted with Temporal Granule Pattern Mining.

John Wesley Hostetter, Mark Abdelshiheed, Tiffany Barnes, Min Chi

May, 2023 In AAMAS

A Self-Organizing Neuro-Fuzzy Q-Network: Systematic Design with Offline Hybrid Learning

Self-organizing neuro-fuzzy Q-networks leverage hybrid learning to produce effective and interpretable policies, which aids human-in-the-loop for design or explainability.

Recent Publications

Quickly discover relevant content by filtering publications.

John Wesley Hostetter, Cristina Conati, Xi Yang, Mark Abdelshiheed, Tiffany Barnes, Min Chi (2023). XAI to Increase the Effectiveness of an Intelligent Pedagogical Agent. In IVA.

PDF Cite DOI

John Wesley Hostetter, Min Chi (2023). Latent Space Encoding for Interpretable Fuzzy Logic Rules in Continuous and Noisy High-Dimensional Spaces. In IEEE.

PDF Cite

John Wesley Hostetter, Mark Abdelshiheed, Tiffany Barnes, Min Chi (2023). Leveraging Fuzzy Logic Towards More Explainable Reinforcement Learning-Induced Pedagogical Policies on Intelligent Tutoring Systems. In IEEE.

PDF Cite

Mark Abdelshiheed, John Wesley Hostetter, Tiffany Barnes, Min Chi (2023). Bridging Declarative, Procedural, and Conditional Metacognitive Knowledge Gap Using Deep Reinforcement Learning. In CogSci.

PDF Cite

Mark Abdelshiheed, John Wesley Hostetter, Tiffany Barnes, Min Chi (2023). Leveraging Deep Reinforcement Learning for Metacognitive Interventions across Intelligent Tutoring Systems. In AIED.

PDF Cite

See all publications