Reinforcement Learning

Question

I want to use this q-learning (reinforcement learning) code. It seems like the code is correct, but I am getting errors and I don't know why:

function  q=ReinforcementLearning 

clc;
format short;
format compact;
int state=0;


R= [-inf,-inf,-inf,-inf,   0,-inf;
-inf,-inf,-inf,   0,-inf, 100;
-inf,-inf,-inf,   0,-inf,-inf;
-inf,   0,   0,-inf,   0,-inf;
   0,-inf,-inf,   0,-inf, 100;
-inf,   0,-inf,-inf,   0, 100];

gamma=0.8;

q=zero(size(R));
q1=one(size(R))*inf;
count=0;

for episode = 0:20000;
y=randperm(size(R,1));
state=y(1);

x=find(R(state,:)>=0);
if size(x,1)>0,
  x1=RandomPermutation(x);
  x1=x1(1);
end

qMax=max(q,[],2);
q(state,x1)=R(state,x1)+ gamma* qMax(x1);
  int  state=x1;


if sum(sum(abs(q1-q)))<0.0001 && sum(sum(q>0))
  if count > 1000;
      
      break
  else
      count=count+1;
  end
else
  q1=q;
  count=0;
end

end

---

But I am getting the following warning and error:

Warning: The method char/int will be removed in a
future relase. Use sym/int instead. For example
int(sym('x^2')). 
> In char.int at 10
  In ReinforcementLearning at 6 
Error using mupadmex
Error in MuPAD command: Invalid integrand. [int]
Error in sym/int (line 107)
   rSym =
   mupadmex('symobj::intindef',f.s,x.s,options);
Error in char/int (line 12)
y = int(sym(f),varargin{:});
Error in ReinforcementLearning (line 6)
int state=0;

---

This code can be found in the following link: http://people.revoledu.com/kardi/tutorial/ReinforcementLearning/Q-Learning-Matlab.htm

Answer 1

There is no

int state=0;
int state=x1;

in matlab. That is C style. In matlab int is a built-in function meaning something else. Besides, it should be

q=zeros(size(R));
q1=ones(size(R))*inf;

Remember to download his RandomPermutation function, otherwise just use randperm instead.