copy contents of an IFRAME to a varible

Hi all
I'm looking for a way to copy the contents on an iframe to a variable programmatically
I have been Successfully using a program to scrape a website for stock market quotes for the past few years and
only for personal use , this has come to a screaming halt in the last week as the site seems to either block my attempts
or uses another method of displaying the table data that is not accessible via httpget

I have managed to get around this by using an iframe to display the required data page within my RB program and then right click to copy and the paste
the contents into a textarea. stripping out the info I require is quite easy.

I then deal with the results in my calculations.

I would like to automate this.

I have read of ways to do it with javascript however its a little out of my league at the moment.

any help appreciated.

cut down code:
global data2save$
dim Opt$(1000): Year$="2017" : asxcode$="xjo" : Month$="Jun"
'print DefaultDir$ + "/public/test.txt"
'create a button with sub event handler
'goto [skip]
button #doStuff, "Paste data then press Button ", [savethefile]
print:print
html"<left>"
textarea #t, "",185,10


url2find$="http://www.asx.com.au/asx/markets/optionPrices.do?by=underlyingCode&underlyingCode="+asxcode$+"&expiryDate="+Month$+"+"+Year$+"&optionType=B"
print url2find$
''html "<div id='IFR' ALIGN=CENTER><IFRAME SRC="+url2find$+" WIDTH=600 HEIGHT=400 ></div>"
'html "<div id='IFR' ALIGN=RIGHT><IFRAME SRC='"+url2find$+"' WIDTH=200 HEIGHT=800 ></IFRAME></div>"
html"<iframe src="+url2find$+" width=80% height=600 align=left></IFRAME>"
#t setfocus()
print


wait

[savethefile]
data2save$ = #t contents$()
call tableselect
#t text("")
CLS
'textarea #t2, data2save,200,30ドル
'print #t , data2save$
open DefaultDir$ + "/public/test.txt" for output as #f
data2save$= strRep$(data2save,ドル",","")
print #f, data2save$

close #f
[skip]

Call setCSS

OPEN DefaultDir$ + "/public/test.txt" for input as #f
z=1
WHILE eof(#f)=0
line input #f, Opt$
' need to load up the file into an array
' then insert the comma between field after
if Opt$ <> "" then
token$ = "?"
for index = 1 to 10
token$ = word$(Opt,ドル index)
newline$=newline$+","+token$

Opt$(z)=newline$
next
' print Opt$(z)
z=z+1

end if

newline$=""
WEND

close #f

for z = 3 to z-1
length= LEN(Opt$(z))
Opt$(z)= RIGHT$(Opt$(z),length-1)
length= LEN(Opt$(z))
if right$(Opt$(z),1)="," then
Opt$(z)=LEFT$(Opt$(z),length-1)
if RIGHT$(Opt$(z),5)=",,,,," then
Opt$(z)=Opt$(z)+","
end if
end if
'Opt$(z)= Chr$(34)+Opt$(z)+Chr$(34)
Opt$(z)= strRep$(Opt$(z),",",";")

'print Opt$(z)
next







DIM tabledat$(11,z+20): dim token$(11) 'unsure why I need the extra 20

' SORTED THE ZERO BASE for tabledat$(x,x)
' so the link's would work
for column =1 to 10
for z= 3 to z+2
item$ =word$(Opt$(z), column, ";")
tabledat$(column-1, z-3) =item$ ' to avoid taking in the first 3 lines (header)
next z
next column

' TABLE HEADER

html"<center>"
div mytable
table #mytable,tabledat$()
html "<table align = center border=1>"

#mytable columnnames("Code,Expirydate,P/C,Exercise,Bid,Offer,Last,Volume,Openinterest,MarginPrice")
#mytable link("Code", "[order]")
#mytable caption("POTENTIAL TRADES")
cssclass "table", "{ border: 3px ridge #DDF ; border-spacing: 4px }"
html "</table>"
render #mytable

end div








wait
[order]
print "You selected "; EventKey$
wait
wait

sub tableselect
data2save$= upto$(data2save,ドル"An *")
data2save$= after$(data2save,ドル"Code Expiry")
end sub

function after$(source,ドル match$)
i = instr(source,ドル match$)
if i then after$ = mid$(source,ドルi+len(match$))
end function


function upto$(source,ドル match$)
i = instr(source,ドル match$)
if i then upto$=left$(source,ドルi-1) else upto$=source$
end function

FUNCTION strRep$(str,ドルrep,ドルwith$)
ln = len(rep$)
ln1 = ln - 1
i = 1
while i <= len(str$)
if mid$(str,ドルi,ln) = rep$ then
strRep$ = strRep$ + with$
i = i + ln1
else
strRep$ = strRep$ + mid$(str,ドルi,1)
end if
i = i + 1
WEND
END FUNCTION


Sub setCSS
CSSID #Container, "{
align center
margin-left: auto;
margin-right: auto;
width: 70%;
Margin: 20px Auto;
Background-Color: #C6C6B8;
Font-Family: Tahoma;
Font-Size: 1em;
Font-Weight: Bold;
Text-Align: Center;
Color: Black;
Width: 1000px;
Height: 710px;
Padding: 20px 20px 0px 10px;
Border-Color: #45453D;
Border-Size: Thick;
Border-Style: Ridge;
Border-Bottom-Style: Ridge;

}"
cssid #navigation, "{
font-family: Tahoma;
font-size: small;
color: #CC0000;
border-right: 3px solid #666666;
background: #ADAD99;
padding: 4px;
width: 250px;
height: 570px;
float: left;
white-space: normal;
background-repeat: repeat;
background-image: url('/pagebg.PNG');
Text-Align: Center;
}"
'********************************************************



CSSID #footer, "{
Background-Color:#77776E;
Font-Family: Verdana;
Width: 999px;
Height: 25px;
Padding: 5px 5px 5px 0px;
Text-Align: Center;
}"

CSSID #mytable, "{
align center
border: 3px ridge #DDF ;
border-spacing: 4px
Font-Family: Verdana;
Font-Size: .9em;
Padding: 10px 10px 0px 0px;
Width: 800px;
Height: 570px;
overflow-y: scroll;
}"
cssclass "tr", "{ background: #CCF }"


CSSID #h1, "{
font-family: Tahoma;
display: block;
font-family: Tahoma;
color: #3D3D1F;
font-size: 2em;
margin-top: 0.67em;
margin-bottom: 0.67em;
margin-left: 0;
margin-right: 0;
font-weight: bold;
}"
End Sub

Hmmm! Maybe I'm doing something wrong. But when I run the program I get the following URL. I enter it into the browser it says the site cannot be reached.
http://www.asx.com.au/asx/markets/optionPrices.do?by=underlyingCode&underlyingCode=xjo&expiryDate=Jun+2017&optionType=B

Is this URL correct..

Why not strip the html stuff off the web page and use it directly instead of a copy and paste.
Seems like it would be a lot easier??
But I'm not sure what you are trying to do?

Dan

thanks for trying Dan
yes the link is correct, I have been scraping data from the site for years with no problems (using code you shared on the RB Wiki.
but as of last week the code does not work... well the code works does but the data that httpget$ receives does not include a <table>
none of my LibertyBasic programs work either.

so as stated above I have gotten around this by copy and paste but its slow when you look at 30 or 40 stocks at a time.

I've spent the last couple of hours trying to understand how to trigger the Javascript function below

button #doStuff, " Press to Copy ", [copythepage]
url2find$="http://www.asx.com.au/asx/markets/optionPrices.do?by=underlyingCode&underlyingCode=xjo&expiryDate=May+2017&optionType=B"
html"<iframe name= anIFrame src="+url2find$+" width=80% height=600 align=left></IFRAME>"




wait
[copythepage]
cls

Html "<Script Type='Text/Javascript'>"
html"function getContentFromIframe(iFrameName)
{

var myIFrame = document.getElementById(iFrameName);
var content = myIFrame.contentWindow.document.body.innerHTML;
return content
}"
Html "</Script>"
print getContentFromIframe(anIFrame)

wait

I see the problem. That site is executing JS code when it sends the page. So the htmlget$ statement won't work. And as you found out you have to let it display somewhere like in the <iframe. You may also want to try the <a href= type.
Have you tried something like this.
I put the id='ifa' in a <DIV> before the <iframe> and all i get is the <iframe statement. Maybe you can move the id somewhere else to get it..
button #doStuff, " Press to Copy ", [copythepage]
url2find$="http://www.asx.com.au/asx/markets/optionPrices.do?by=underlyingCode&underlyingCode=xjo&expiryDate=May+2017&optionType=B"
'url2find$="http://rfo-basic.com/"

html "<textarea id='ta'>..</textarea>"
html"<div id='ifa'><iframe id='if' src="+url2find$+" width=80% height=600 align=left></IFRAME></div>"

wait
[copythepage]
html "<Script Type='Text/Javascript'>
document.getElementById('ta').value = document.getElementById('ifa').innerHTML;
</script>"
a$ = #request get$("ta")
print "======>";a$
wait

If that don't work, let me know.. Maybe there is another way...

G'day Dan,
thanks for taking a look at this again
running your code the textarea receives

and not the actual contents of the iframe

I have a working version using "copy" & "paste" , I'll keep pugging away trying to automate it

Avagoodweegend

Laurie

translation:
here